The QICKD study protocol: a cluster randomised trial to compare quality improvement interventions to lower systolic BP in chronic kidney disease (CKD) in primary care
© de Lusignan et al; licensee BioMed Central Ltd. 2009
Received: 11 February 2009
Accepted: 14 July 2009
Published: 14 July 2009
Chronic kidney disease (CKD) is a relatively newly recognised but common long-term condition affecting 5 to 10% of the population. Effective management of CKD, with emphasis on strict blood pressure (BP) control, reduces cardiovascular risk and slows the progression of CKD. There is currently an unprecedented rise in referral to specialist renal services, which are often located in tertiary centres, inconvenient for patients, and wasteful of resources. National and international CKD guidelines include quality targets for primary care. However, there have been no rigorous evaluations of strategies to implement these guidelines. This study aims to test whether quality improvement interventions improve primary care management of elevated BP in CKD, reduce cardiovascular risk, and slow renal disease progression
Cluster randomised controlled trial (CRT)
This three-armed CRT compares two well-established quality improvement interventions with usual practice. The two interventions comprise: provision of clinical practice guidelines with prompts and audit-based education.
The study population will be all individuals with CKD from general practices in eight localities across England. Randomisation will take place at the level of the general practices. The intended sample (three arms of 25 practices) powers the study to detect a 3 mmHg difference in systolic BP between the different quality improvement interventions. An additional 10 practices per arm will receive a questionnaire to measure any change in confidence in managing CKD. Follow up will take place over two years. Outcomes will be measured using anonymised routinely collected data extracted from practice computer systems. Our primary outcome measure will be reduction of systolic BP in people with CKD and hypertension at two years. Secondary outcomes will include biomedical outcomes and markers of quality, including practitioner confidence in managing CKD.
A small group of practices (n = 4) will take part in an in-depth process evaluation. We will use time series data to examine the natural history of CKD in the community. Finally, we will conduct an economic evaluation based on a comparison of the cost effectiveness of each intervention.
Clinical Trials Registration
ISRCTN56023731. ClinicalTrials.gov identifier.
Chronic kidney disease (CKD) is a common long-term condition, affecting 5 to 10% of the population. CKD is an independent risk factor for cardiovascular disease, established renal failure (ERF) and all cause mortality [1–3]. Patients with CKD are far more likely to die prematurely from cardiovascular disease than progress to ERF requiring dialysis or transplantation. The presence of proteinuria confers additional cardiovascular risk.
CKD is classified into five stages based upon a measurement of kidney function and the estimated glomerular filtration rate (eGFR) determines the class of CKD for the more severe stages (Stage three to five). Stage one and two are the mildest of the five stages of CKD and require evidence of kidney damage, usually the presence of proteinuria, to confirm the diagnosis. Stages three to five CKD can be diagnosed by eGFR alone; and stage three is now often split into stages 3a and 3b, as there are far higher rates of cardiovascular co-morbidity in stage 3b disease. People with cardiovascular co-morbidities especially hypertension and diabetes; cardiovascular risk factors, particularly raised systolic blood pressure (BP); and more specific renovascular risk factors: proteinuria and anaemia are at increased risk.
There is a broad and evidence-informed consensus that lowering BP is of central importance, both to slow the progression of CKD and reduce cardiovascular risk. Lowering of BP using angiotensin modulating anti-hypertensives, angiotensin converting enzyme inhibitors (ACEI) and angiotensin (II) receptor blockers (ARB) appears to have additive renal-protective benefits . Strict management of BP, cardiovascular and specific renovascular risk should be feasible in primary care. Guidelines on the management of CKD have recently been published by the National Institute for Health and Clinical Excellence (NICE) . In the absence of proteinuria, the threshold for intervention is a BP of ≥ 140/90 mmHg is recommended, with a target systolic BP of between 130 and 139 mmHg. In diabetes and where significant proteinuria is present, the respective values are 130/80 mmHg with a systolic target of between 120 and 129 mmHg. However these targets frequently remain unmet. Studies have demonstrated a need to improve both information and training available to practitioners with the aim of enabling them to improve the quality of care currently provided .
There is limited knowledge and experience of managing this condition in primary care, and while CKD has been included as one of the financially incentivised chronic disease management targets for general practice – the 'Quality and Outcomes Framework' (QOF) it is the only QOF indicator to be accompanied by a 'Frequently Asked Questions' document – requested by the British Medical Association as a condition for the inclusion of this indicator in the QOF indicator set . Feedback to the investigators has been that practitioners lack confidence in the management of this condition, especially implementing the BP targets in elderly patients (who are at higher risk of CKD and its sequelae).
There are further problems with the QOF. The use of routinely collected clinical data for purposes other than clinical care may distort data recording . Practitioners feel reluctant to include a patient with incomplete data on a QOF disease register as this might affect their income. Regardless, the prevalence of CKD reported through the QOF to the NHS Information Centre for 2006/7  is less than half that reported in the epidemiological studies quoted in this introduction. There is de facto a quality gap as those people with CKD not on the disease register will not be recalled for BP and other checks.
Finally, the new NICE guidance looks at CKD at a point in time . Management is largely determined by the eGFR over a three-month period, BP control and the presence or absence of proteinuria. Although there is a heuristic for a rate of decline that would trigger referral, there is dissonance between this heuristic and clinical practice in primary care. Many elderly people with CKD, even more advanced stage four disease, appear to be stable and the NICE along with previous guidance may be over aggressive for this group of patients; this may be part of the reason why clinicians are not implementing recommended BP targets [9, 10]. Further research is needed to understand the natural history of the disease and whether rate of decline would be a more appropriate primary variable to detect people at risk.
The quality of general practice computer data
UK general practice is almost universally computerised and has some of the most advanced general practice computing [11, 12]; providing a rationale for the use of routinely collected data to measure the impact of the quality improvement interventions being developed and tested in this programme of research. Six factors contribute to the high quality of general practice computer data: we have an accurate denominator ; prescribing records are largely complete; electronic connections to laboratories mean that pathology data are complete; the QOF has improved data quality in CKD and its cardiovascular co-morbidities including diabetes; an electronic referral system has improved data quality; and the NHS has sponsored the development of a tool called MIQUEST (Morbidity Information Query and Export Syntax) to extract anonymised data – a tool we have over 10 years experience of using [14, 15].
Optimal management of CKD in primary care is currently limited by a lack of knowledge about how to increase adherence to guidelines for best practice . There is no single perfect quality improvement strategy to use in primary care . The most commonly used strategy is dissemination of clinical practice guidelines with prompts . This usually involves distribution of paper guidance and reminders with internet resources providing additional information and support. More expensive and complex interventions have been widely used, including audit-based education (ABE) where practitioners compare their own practice's adherence to guidance with that of peer practices [19, 20]. Our experience from observational work has been that ABE is more effective in its second year ; a similar pattern is seen with using feedback to improve data quality .
Study aims and objectives
This study aims to improve the quality of CKD management in primary care with the emphasis on strict control of systolic BP to reduce cardiovascular risk and slow renal progression.
To lower the BP of hypertensive individuals with CKD to an agreed target.
To measure the impact of the quality improvement interventions on the recording and control of renovascular risk factors, including proteinuria; and cardiovascular co-morbidities, including diabetes mellitus.
To evaluate the quality improvement interventions and measure their impact on other markers of quality, including practitioner confidence.
To establish a cost model for each quality improvement intervention.
To characterise the natural history of CKD. We wish to compare those who have progressive (as defined by a yearly decrease in eGFR of >5 ml/min/1.73 m2 in one year or >10 ml/min/1.73 m2 in 5 years) , compared with non-progressive renal disease; comparing demographics, co-morbidities (including diabetes), and biomedical variables.
To develop improved primary care guidelines for management of CKD and measure adherence to this guidance; with an emphasis on comparing progressive, with non-progressive CKD.
Study design overview
We plan to conduct a two-year, three-arm cluster randomised trial. We are carrying out a cluster randomised trial because we feel that quality improvement is often adopted at the level of the practice. A trial of individual patients would be much more difficult because it may be impossible to stop contamination between general practitioners (GPs) and other health professionals working in the same practice; GPs may see successive patients from different arms of the trial; and communication between patients randomised to different arms of the trial might also bias results.
Four in-depth process evaluation practices: These practices will take part in our diagnostic analysis process at the start of the study proper (i.e., does the intervention meet their perceived needs, and does it address barriers to quality improvement). They will validate our questionnaire to assess confidence and, during the study proper, report on the intervention exposure (i.e., to what extent the intended recipients are exposed to the interventions); and programme fidelity (i.e., whether the quality improvement intervention is delivered as planned). Two practices will give in-depth feedback about the GaP intervention and two about ABE. We will use focus groups run in each practice as our principal method of data collection; however we also plan a mid-study workshop of all the in-depth process evaluation practices. Our sample will include at least one practice from the north and from the south; we intend to recruit inner city, suburban, and county town practices; we want to see the four major brands of general practice computer systems represented across the practices so that we can also test our queries and data extracts.
An additional 10 practices in each arm will complete a confidence questionnaire: We will recruit 10 additional practices in each arm that will participate in the study but also complete a questionnaire about their confidence in the management of CKD. We are primarily doing this to assess if any of the interventions have a greater effect on confidence. We are sending this questionnaire to a separate group of practices because completing the questionnaire may be an intervention in its own right, possibly as great as GaP. We will be able to compare questionnaire and non-questionnaire practices in each arm at the end of the study.
The participants are GPs located in practices (our clusters) across England. We aim to recruit a nationally representative sample of practices from: in and around London – especially inner city and suburban southwest London; urban and rural Surrey and Sussex; Leicester city and surrounding areas; Birmingham inner city and suburban; and Cambridge. The locality structure is pragmatic because groups of practices need to come together for the ABE workshops. An inclusion criteria for a locality is that their local renal unit would support the workshop within their locality and review the GaP to minimise any conflict with local policy.
The primary research participants are GPs involved in the study who will receive the quality improvement interventions listed below. The interventions will be implemented at the practice (cluster) rather than the individual level. The study subjects (who may be regarded as secondary participants) will be all individuals with CKD within the study practices. CKD will be defined using the internationally accepted National Kidney Foundation (NKF) definition  using two measures of eGFR of less than 60 ml/min/1.73 m2 at least three months apart. However, we will also explore the effects of including people with a single recording of eGFR.
The participants do not receive any financial incentives to participate, though they do receive financial compensation for the time actually spent attending study activities. These will vary according to the arm of the study they are allocated to.
Inclusion and exclusion criteria
Localities require the local renal unit to share local guidance and support our interventions.
Primary care organisation approval for the research to be conducted in their locality.
Practices who provide written consent to participate.
Agreement to participate in whichever arm of the study they are randomly allocated.
Practice has had the same computer system for the last five years and has no plans to change it, and will allow access to check data quality.
Practice has electronic laboratory links for three years or more.
Practices in whom the computing system has changed over the last five years.
Practices lacking an appropriate computer system from which data can be extracted.
Practices in which referral data (from primary care to secondary care) is not available.
Practices planning to move computer system in the next two years.
Dedicated members of the study team (NT and NJ) liaise with and recruit eligible practices from the study's 'localities' who meet with the above inclusion criteria. The primary care research networks, funded by the National Institute for Health Research (NIHR) have actively supported the recruitment for the study in all of our target areas since the project was added to the NIHR portfolio of research projects. Recruitment has also been carried out by writing to practices associated with teaching networks in southwest London, Surrey and Sussex (SdeL). There has been word-of-mouth recruitment from members of the project team, and snowball recruitment from practice to practice.
Practices will be asked to consent as a unit, with all GPs being willing to participate. One or more persons will sign the consent form as authorised by the practice. This may vary from all GPs to one GP being authorised to consent on behalf of the practice. No direct consent is taken from patients, however a waiting room poster is provided as well as a lay summary of the project in leaflet form.
The interventions in the study
Two interventions are being compared to usual practice: GaP and ABE. The interventions are designed to target the cluster (i.e., individual general practices). Where we send GaP or questionnaires we send them to individual named clinicians. Where a practice is invited to attend an ABE workshop all members may attend; however, our experience is that one or more practice members attend on behalf of the others; we try to compensate for this by providing learning resources for them to take back to their practices. However, although we send some material to individuals, the intervention is focused at the level of the practice.
These practices will be allocated to this arm at randomisation (n = 35 practices – 25 in the core CRT and 10 in the questionnaire group). Once assigned to this arm, a minimum of contacts will be made of these practices other than for data collection.
Distribution of clinical practice guidelines with prompts (GaP)
This is an established, low cost method of quality improvement . It will provide a benchmark with which the effectiveness of the other quality improvement intervention can be compared. We will develop a consensus between the study team, our expert advisory group, and external peer reviewers, and produce appropriate guidance for the management of CKD in primary care. This guidance will be distributed to practices within this arm of the CRT (n = 25 practices plus 10 questionnaire practices) with six monthly updates and reminders. The guidance will be customised to fit with local practice and reflect guidance in that area. In addition practices will have access to a supportive website with information about CKD, frequently asked questions, and tools to improve CKD management.
The GaP documentation will typically be up to four sides of A4 paper stock, published in a glossy professionally printed form. It may be accompanied by local guidance or national brief guidance in the first intervention. We plan to distribute the NICE 'Quick Reference Guide' to managing CKD  as part of the second-year intervention.
Audit based-education (ABE)
In this arm, practices (n = 25 practices plus 10 questionnaire practices) will have a representative attend workshops. These practices will also have access to clinical practice guidelines provided to the second arm of the study. However, in addition, practices will receive three sets of detailed comparative feedback about their quality of CKD management at approximately nine-month intervals, and we will facilitate lists of patients needing interventions (local queries) being created within the practice. This comparative feedback about adherence to guidelines will be based on anonymised data collected from their general practice computer system prior to the ABE workshop.
The study will use an ABE model for quality improvement developed by the primary care data quality project that has been used in a variety of clinical contexts . This involves feedback given in a workshop setting with at least one GP and one nurse or practice manager from each practice present. The workshops will be in two parts: the first will be facilitated by a GP familiar with the data, ideally from the locality, but if not, available from study team, and a local renal specialist in attendance to provide expert advice and information about local practice. The first part will be a presentation of the comparative adherence to evidence-based guidance for the management of CKD by the different practices present led by the GP. This section will highlight variation in the quality of care in a non-judgemental context. The second part of the meeting will be case studies, facilitated by the local consultant, which small groups will work through to explore dilemmas in management and how to overcome them.
The workshops are timetabled for two and a half hours of activity with additional break time to allow informal contact. Practices are expected to bring along at least one GP and one or two other members of the practice team: their practice manager and a nurse involved in cardiovascular risk assessment or diabetes within the practice.
Delegates are asked to fill in a feedback form, of the standard type used to evaluate educational meetings, on the usefulness and appropriateness of the content and the educational methods used. There is also the opportunity to provide informal feedback. This feedback, along with a narrative from the three members of the study team who participate in these workshops (it is expected there will be at least three) will be fed back into the design of subsequent feedback. Semi-structured interviews – reviewing the appropriateness of the level; the content and the delivery are being held in person or by telephone with all members of the study team who had attended or participated in the first round of workshops.
The content of the interventions
The content and focus of the GaP arms of the study will be the same as in the ABE arm. The areas and learning objectives for each year have been set; however, the specific details will depend on the national guidance available at the time. Currently, we are basing our year one criteria and standards on the NICE guidance released in September 2008 .
During the first year, the clinical focus will be on understanding any gap between the 'true' prevalence revealed by the audit and the 'QOF prevalence' the practice reported to the NHS Information Centre, which is publicly available information . We expect our audit to identify approximately double the number of people with CKD than included in the practice QOF disease register. In addition, this year will look at proteinuria recording, control of BP and use of appropriate therapy: angiotensin modulating drugs, appendix 1.
The second year's clinical focus will be on the management of co-morbidities, especially diabetes. Strict control of cardiovascular risk factors in patients with CKD and Cardiovascular System (CVS) risk is important. We also look at control of BP in diabetes. People with diabetes and CKD need stricter BP control, especially if they have microalbuminuria; diabetics are also one of the most likely groups to go on to require renal replacement therapy, appendix 2.
What happened: Clinical outcomes and change in practitioner confidence.
Why change happened: Diagnostic analysis plus process evaluation.
What it cost: Economic evaluation.
Primary outcome measure
The primary outcome measure is the reduction of systolic BP in hypertensive people with Stage three to five CKD towards the current national target . [Hypertension is defined as above >140 mmHg in low-risk patients and >130 mmHg in high-risk patients. High-risk patients are people with CKD plus significant proteinuria (ACR ≥ 70 mg/mmol; or equivalent) or with CKD and diabetes.
We plan to measure the effect of the intervention across the same cohort, though we recognise that it will have less effect on people in stage four and five CKD, as these people are largely managed by specialists. However, as they represent a small percentage of the people with stage three to five disease (<5%) , we think this is unlikely to significantly distort our results. We will also explore the effect of the intervention on people older than 75 years.
Secondary outcome measures
In addition to measuring the effect of the various quality interventions upon systolic BP, will study a number of secondary outcomes:
Clinical and laboratory markers
Case definition using eGFR: We will define cases using the internationally accepted definition used by NKF Kidney Disease Outcomes Quality Initiative (KDOQI) , the same definition is used by NICE . We will identify cases recorded since the standardisation of creatinine recording in 2006. However, we will also undertake a sensitivity analysis, including how prevalence changes when the date or number of readings are changed.
BP: We will measure the proportion of people in each arm with hypertension and CKD who achieve at least a ≥ 5 mmHg reduction in systolic BP. The reduction of mean systolic BP (the primary outcome measures) could be distorted in a number of ways.
Recording and management of key co-morbidities: diabetes and its complications; ischaemic heart disease; heart failure; obstruction/lower urinary tract symptoms.
Recording and management of other cardiovascular risk factors: smoking status; lipid management; proteinuria; anaemia; glycated haemoglobin and microalbuminuria in people with diabetes.
Serial measures of serum creatinine concentration and eGFR: to explore natural history and look for cases of accelerated decline (defined as a reduction of eGFR of >5 ml/min/1.73 m2 in one year; or >10 ml/min/1. 73 m2 in five years) .
Recording of death and cause of death: Although this is incompletely recorded, we will attempt to capture any recording as we expect mortality among hypertensive people over the period of the study. There may be a higher mortality among those who are in the control than intervention arms.
Avoiding harm: We wish to monitor whether BP reduction is associated with an increased number of falls particularly in older people. Most people with CKD are elderly and at potential risk for falls. Notwithstanding the results of recent systematic reviews that failed to show an association between falls and anti-hypertensive medication [25, 26], this remains a genuine concern to some practitioners, and one that we propose to examine. A falls dataset will be devised and integrated into the renal dataset. We will investigate the relationship with use of ACE inhibitors and angiotensin II receptor blockers and systolic BP below 120 in CKD.
a. Use of drugs/therapy that affect renal function (for example non-steroidal anti-inflammatory drugs)
b. Use of ACEI and angiotensin II receptor blockers to control hypertension
c. Recording of medicines possession ratio based on days prescribed therapy as an index of concordance with anti-hypertensive therapy.
The details of our dataset are shown in appendix 3.
Diagnostic analysis and process evaluation, including confidence and end of project questionnaires
Practitioner confidence to be measured at t = 0, t = 18 months using a questionnaire that assesses confidence.
Feedback from focus groups held prior to round one (diagnostic analysis).
Feedback from focus groups held mid-study and at the end of the study.
End of study questionnaire and workshops.
We know the economic impact of implementing guidance in place prior to the publication of NICE guidance in September 2008 for the primary care management of CKD . We will update the model used by Klebe et al. to reflect the restriction of investigations for renal bone disease in current guidance  compared with those advocated in previous guidance . We will then compare the projected investigation cost with the true costs as represented in the routinely collected data.
We wish to capture any unintended consequences through our process evaluation arm, especially via the open questions in each year of the study (appendices 1 and 2). Many implementations of IT-based change have unintended consequences . Specifically, we will explore with process improvement practices any issues about calling in or recalling patients, and any adverse reactions to therapy or interactions; we will also look at the rates of collection of prescriptions for ACEI and ARB as a proxy for medicine possession ratio. Quality improvement strategies based on open sharing of data may also have unintended consequences [30–32]; though in this study our data sharing is largely within the peer group rather than with the public.
Data quality assurance
The study has been designed and will be reported in accordance with the CONSORT (Consolidated Statement of Reporting Trials) and its extension to cluster randomised trials . Data will be controlled in accordance with data protection legislation, institutional protocols of St. George's University of London, and NHS policies for research and information governance for ensuring patient confidentiality . Data will be analysed in SPSS (Statistical Package for Social Sciences) version 15 using an intention to treat approach.
These data will be extracted from general practice computer systems using the department of health sponsored data extraction system MIQUEST. MIQUEST has been developed by the NHS and is used in the national data quality programme at PRIMIS (Primary Care Information Services) . This application allows identical searches on different brands of general practice computer systems. MIQUEST, when written in its 'remote' mode, extracts pseudo-anonymised clinical data. In its 'local' mode, it allows the extraction of patient identifiable data, such as postcodes for mapping onto multiple deprivation index, and for case-finding within the practice.
Routinely collected general practice computer data are complex and require significant processing and interpretation in order to obtain meaningful information . The research team has considerable experience and has developed a published method . The research data will be completely traceable due to the development of a sophisticated meta-data schema [38, 29]. Our extraction technique includes thorough piloting and planning, and data processing with quality controls at each stage. All variables are examined for their distribution, and cleaned appropriately. Where possible, we use therapy and/or pathology tests to triangulate diagnostic and symptom codes.
An issue with routine data is that they are incomplete, and in contrast with other trial data are not systematically recorded at regular intervals. However, we expect to have relatively complete data on people with cardiovascular co-morbidity for the last five years (since the 2004 new contract for general practice) and hopefully longer. The quality of UK primary care data continues to increase, and there is a growing amount of published research that is based on routinely collected data – especially from countries with registration based primary care .
We have an agreement with CKD researchers in Galway, Ireland, who have experience of using routinely collected data to research CKD [39, 40], that they will independently scrutinise our analysis procedures and generation of results tables.
Diagnostic analysis and process evaluation
The questionnaire to test practitioner confidence has been developed using a standard questionnaire development method . This questionnaire, developed by GP experts and renal specialists, has been validated through initial testing within the study team, then tested within a south London practitioners group who are not participants in this study. Finally, it was tested within our process evaluation group. The questionnaires are sent to individual health care professionals participating in this study; they are numbered so that reminders can be sent and survey data at the different time points can be inked. Reminders are sent by post. There will also be a final reminder by telephone.
The focus groups are led by members of the study team after receiving training from an experienced qualitative researcher, IC. The focus groups are recorded and transcribed verbatim before IC undertakes more detailed analysis. The analysis will utilise the 'framework' approach developed at the National Centre for Social Research and now a widely used method for analysis within the field of health and social care research . The emergent themes will be discussed with the study team. Focus groups will be continued until thematic saturation is reached.
The Health Foundation is providing expert health economic consultancy to the quality improvement projects. Once our first-round data collection is complete, we will review this with the expert advisors .
Cluster randomised trial sample size
SK, an experienced medical statistician with specific expertise in cluster randomised trial design [44, 45], conducted a sample size calculation taking into account variation between practices. The study is powered to detect a >3 mmHg difference in systolic BP between the groups over the two-year duration of the study. Because of the large number of patients per cluster, the sample size can be estimated using a 'summary statistic' approach whereby each practice provides a single mean BP. Using a sample dataset of 30 practices, we have estimated that the variation between practice means has a standard deviation (SD) of 3.77 mmHg. Assuming that this sample of 30 practices is representative of the study practices in terms of their size and number of CKD patients, a sample size of 25 practices per intervention group will be required to detect a difference of 3 mmHg at the 5% level with a power of 80%.
A larger difference of clinical importance (e.g., 5 mmHg) would require a smaller sample. However, given the population nature of this intervention, we decided to be prudent and power the study for a small difference.
A sample of 10 practices in three arms should enable us to compare changes in confidence in managing CKD. We expect to recruit practices with a mean practice list size of around 8,000 . The latest workload survey suggested that 62% of GPs work full time . There is approximately one GP per 1,700 patients. The confidence questionnaire adopts a five-point scale.
We estimate that there will be at least two practice nurses per 8,000 patients engaged in assessment of cardiovascular risk including management of CKD. We estimate an average of 10 practitioners per practice are eligible to complete the questionnaire and that we will achieve a >60% response rate, or 180 returned questionnaires.
A pilot study as part of the development of the questionnaire shows that the responses have a mean score of two, and standard deviation of about 1.26. We want to have a power of 0.80, or equivalently, the probability of a Type II error of 0.20, the sample size needed to show a change of 0.5 units in the five-point scale, the smallest individual change meaningful for the study, is 33 practitioners in each arm of the study.
Although negative effects are unlikely, any suspected negative effects will be investigated and the study suspended, pending review. The principal safety monitoring activities will be: the observation for falls in people newly started on additional BP lowering drugs; and to identify whether there is any relationship between systolic BP and rate of falls.
Randomisation was conducted in blocks
Practices agree to participate in the study the basis that they will be assigned at random to an arm of the study. We excluded practices who wanted to choose an arm of the study. They are assigned their arm by simple random allocation. Randomisation will be performed with a table of random numbers by JvV; in the order practices complete their consent to participate. He allocates, at random, recruited practices in blocks of nine; accepting that there will be a final block of less than nine.
The allocation is not shared with those who will be involved in the data analysis. The clinical data collected are identical in all three arms of the study, so there should be no clues within these data as to which arm is which. The allocated arm is recorded in our database of practice details that is kept entirely separately from the pseudonymised table of data used for analysis. Within the analysis table the practices in each of the three arms are identifiable for analysis – but there is no labelling of which specific arm any practice is allocated to. Similarly, patient and practice identifiers are pseudonymised, which again makes it harder for the analysts to identify individual arms.
Ten practices in each arm are labelled as having had the questionnaire. The four in-depth process evaluation practices have a separate series of identification numbers so that they can have their data analysed but excluded from the study.
The field team are aware of which practices are in which arm, because they must mail or invite participants to the relevant intervention. However, patient and practice details are pseudonymised. All cleaning and processing of data are carried out on the whole database (i.e., all three arms) simultaneously. We will do this by only revealing the arm allocation variable at the end of the study. We try to minimise access to signature data that would allow the arms of the study to be differentiated. (e.g., if an analyst knew the precise list size of one practice in the study.) However, we only plan to reveal this variable when it is needed for final comparison between arms.
Processed data extracted from GP practices and survey data using questionnaires will be imported onto the SPSS or a compatible software system. The data analysis will be conducted in three stages:
Univariate and bi-variate analyses
We will document the recorded prevalence of CKD, as defined by socio-demographic (e.g., age, gender, ethnicity, deprivation scores).
We will document the level of confidence of primary care practitioners in the management of CKD stage three to five as defined by age, role, and the characteristics of the GP practices.
We will compare the recorded management of CKD Stage 3 – 5 in the participating GP practices with national and local guidelines.
We will document the recorded key co-morbidities of CKD stage three to five (e.g., diabetes, ischaemic heart disease etc).
We will compare the recorded management of key co-morbidities in the participating GP practices with national and local guidelines.
We will document the association between management of CKD using BP medication and falls.
Using analysis of variance (ANOVA) models, compare the mean systolic BP of people with CKD stage three to five in the three arms of the study, before and after the interventions – the primary outcome measure of this study
Using ANOVA models, compare the confidence level of primary care practitioners in the management of people with CKD Stage three to five in the three arms of the study, before and after the interventions
Using multiple regression analyses, explore and quantify relations between independent variables (e.g., known demographics and risk factors, such as smoking status, level of cholesterol, obesity, anaemia and alcohol consumption) and dependent variables (e.g., CKD stage three to five, and diabetes).
Longitudinal data analyses
The temporal dimension of the recorded clinical data collected contemporarily offers an opportunity for analyses of the natural history and the disease course of CKD. The data have an advantage of being free from bias from retrospective recall, and allow the follow-up of the full spectrum of the impact of contributory risk factors on and outcomes for people with CKD. A particular interest is the association between management of CKD, the rate of change of eGFR, falls, and the outcomes of CKD.
This study fills a gap in the literature about how to improve the management of CKD in primary care. This gap is worth filling, because interventions that can be administered in primary care should be able to slow the progression of CKD, and consequently reduce cardiovascular co-morbidity and the need for dialysis and transplantation.
The study is a pragmatic approach to quality improvement (QI) in CKD, and is intended to inform practitioners and the commissioners of care about the cost effectiveness of GaP and ABE in this disease area.
The ethical oversight of quality improvement projects remains a subject of much debate . The study does not mandate any new intervention to be given to patients in participating practices, but rather promotes the implementation of best practice. Personalised decisions to treat patients will be made by individual practitioners in partnership with their patients, as now. Indeed, the primary research participants of the study are the participating practitioners rather than they patients they treat. This distinction has been recognised by the ethics committee that approved the study; our view is that studies of this potential size and impact should be part of the ethical approval process. Strictly, it is only the inclusion of randomisation which meant that this study required UK research ethics approval.
There are some weaknesses in the selection of BP as the primary endpoint; however these effects should be the same in each arm of the study. GPs will commonly check BP a second time if it is raised, but not if it is normal. There can consequently be a tendency for regression towards the mean in people with raised BP that is greater than in those with normal BP. This effect will need to be taken into account in the interpretation of the results. It is possible that people with raised BP will be under-detected.
A further problem with BP is that it tends to be recorded in primary care with marked end digit preference (EDP); i.e., a preference for recording a zero or five as the terminal digit . EDP can make BP measurement a very blunt instrument, and make it harder to detect change. Although there has been improvement (i.e., a reduction) in EDP, especially in people with raised BP or cardiovascular co-morbidities, this remains a significant problem. Although, likely to influence each arm equally, EDP reduces the fidelity of our observations.
Routinely collected data are not like trial data; they are recorded inconsistently and reflect the primary healthcare professional's understanding of the problems presented. The record entries are made within the context of a short primary care consultation; what is recorded in the record is not a neutral act and often has connotations for patients (e.g., 'You told me my kidney blood tests are OK but you have labelled me as having CKD') . We are only extracting coded data, and will not have access to free text data where other key data my lie. For example, 'urine NAD' (NAD = no abnormality detected) – a negative urine stick test may be recorded in the records; but as it has not been coded this test will remain hidden. Similarly, hospital letters and reports where the text has not been coded will also remain invisible to our searches.
Some members of the project team have been involved in the development of ABE as a quality improvement intervention for some time (SdeL, TC, JvV, NH) [19–21]. However, we have no personal stake that we feel will bias the outcome of this trial, and building-in independent scrutiny of the data should help ensure this is a fair test.
There are also a number of external pressures that are influencing the study; the most important are QOF CKD Indicator  and NICE guidance  issued in September 2008. The CKD QOF indicator is progressively being aligned with NICE guidance; and it is possible that these influences may be greater than any effect from the study. However, these are also factors which will equally influence all three arms of the study.
This study should provide useful information about the influence of straightforward quality improvement interventions on the management of CKD; and if they are additive on the influences of financially incentivised QOF and the new national guidelines (NICE). The study will face all the challenges associated with working with routinely collected data, as well as the many confounding factors. We anticipate reporting whether the QI interventions tested have a place in improving the management of CKD.
Prevalence. Prevalence of CKD, and prevalence by age band, for each of stage three to five CKD.
Practice prevalence (from serum creatinine records) compared with:
Population prevalence (from literature)
QOF prevalence (based on business case rules)
Standardised prevalence; deprivation and ethnicity recording
Proteinuria recording. Proportion of CKD patients with proteinuria estimation separately in diabetics and non-diabetics. Proportion of patients in whom proteinuria has been measured with albumin: creatinine ratio (ACR)>30 and >70 mg/mmol in non diabetics.
BP. Indicators of BP control.
Number of measurement in last 12 months
Most recent systolic and diastolic BP
Mean systolic and diastolic in last 6 months
Proportion meeting QOF standard, and NICE targets
Angiotensin blockade in CKD. Use of angiotensin modulating drugs (ACEI and ARB)
Total number of prescriptions
Use in CKD with proteinuria
Cardiovascular co-morbidities. Prevalence, use of 10 year risk scoring.
Process of delivering care. Hints, tips, case-studies of how to achieve change (e.g., All hypertensives and those with CVS co-morbidity have a proteinuria test when having their blood tests.). Which primary care professionals are involved? Shift to primary care management.
Motivation to change care. Is CKD an illness? Are we inappropriately labelling much of the elderly population? Do the biomedical interventions do more good than harm?
Improving the intervention. How could the intervention be improved?
Programme fidelity and intervention exposure. Has the implementation been feasible (programme fidelity) and what proportion of the practice have been interested in the feedback and results (intervention exposure)?
How can the QI interventions be improved? Suggested improvements to the interventions.
Diabetes and CKD. Prevalence of Diabetes and CKD and quality of management (comparing quality of management with QOF and national guidance, including new NICE guidance).
BP recording and control and use of angiotensin modulating drugs
HbA1c recording and value (compared with non-CKD diabetics, controlling for age and gender)
ACR (Albumin Creatinine ratio) in people with diabetes with CKD
Lipid management and use of statins and other medications for 1° and 2° prevention
Use of aspirin as primary and secondary prevention
Cardiovascular co-morbidity. We will look at risk factor management in people with cardiovascular disease, to include: use of lipid lowering therapy; use of aspirin; smoking cessation.
Progression of CKD. We will identify people with rapid progression.
Anaemia and CKD. We will flag people with anaemia who cross current NICE thresholds
Avoiding harm. We will look specifically for any evidence of increased numbers of falls; but are open to other unanticipated harmful consequences of the intervention.
Good ideas. The workshops will also seek to capture any examples of good practice and disseminate them across the group.
Process of delivering care. Any issues of call/recall of patients and concordance with therapy – especially angiotensin modulating drugs will be explored.
Unexpected consequences. We will try to identify any unexpected consequences of the interventions; good or bad.
Overview of the dataset extracted
Number and range of practice members engaged in CKD management
Pseudonymised practice indicator
Postcode (only first part is retained)
Index of deprivation (calculated in each practice from the postcode which is then deleted)
Cause of death & death
Clinical and laboratory
Serial measures of BP
Serial measures of serum creatinine concentration and eGFR
Co-morbid conditions (diabetes and its complications, ischaemic heart disease, heart failure, urinary obstruction)
Cardiovascular risk factors: smoking status; serum cholesterol and total cholesterol: HDL ratio; BMI, alcohol consumption; glycated haemoglobin and microalbuminuria in people with diabetes mellitus; urinalysis and total protein creatinine ratio; haemoglobin concentration
Lower urinary tract symptoms, prostate disease and urological factors which may reduce eGFR
Falls dataset (falls, likely fragility fractures, new diagnosis of osteoporosis)
Medications for optimal management that also impair renal function
Referral (to renal, diabetes, care of the elderly, urological and other specialties)
Number of consultations in primary care
This research programme is supported by two peer-reviewed charitable grants. A three-year grant was awarded by Health Foundation as a part of their Engaging with Quality in Primary Care scheme. Additional support focussed on chronic kidney disease in patients with diabetes has been provided by a separate award from the Edith Murphy Foundation.
Several senior academics have supported the development of this study, and its design. We received important methodological advice from: Professors Sean Hilton, Martin Eccles, Richard Hobbs, and David FitzMaurice. They all advised a change from our original locality based plan to a CRT, where individual practices were the cluster. We have also had extremely helpful CKD related advice from our Advisory Board – especially: John Bradley (Chair), Donal O'Donaghue, Charlie Tomson, Paul Stevens, and Azhar Farooqi, We also acknowledge: Jo Moore, our current project manager and Bernie Stribling who previously held this post; Sally Kelly (SK), a statistician who provided advice about the power calculation; James Hollingshead and East Midlands Public Health Observatory (lead national PHO for CKD) for help with prevalence calculations; Linzie Long, Imran Rafi, Ravi Seyan, who supported the development of the ABE intervention; Nigel Mehdi and Mark Bradley, for expert advice and consultancy to develop and improve the functionality of our data warehouse; Support with our ethics application from Bryony Soper and other members of the Improvement Foundation team funded by the Health Foundation as part of our financial support; The National Institute for Health Research: Comprehensive Research Network (CRN) and PCRNs for supporting this work, especially recruitment into the study.
- Weiner DE, Tighiouart H, Amin MG, Stark PC, MacLeod B, Griffith JL, Salem DN, Levey AS, Sarnak MJ: Chronic Kidney Disease as a risk factor for cardiovascular disease and all-cause mortality: a pooled analysis of community-based studies. J Am Soc Nephrol. 2004, 15 (5): 1307-1315. 10.1097/01.ASN.0000123691.46138.E2.View ArticlePubMedGoogle Scholar
- Go AS, Chertow GM, Fan D, McCulloch CE, Hsu CY: Chronic kidney disease and the risks of death, cardiovascular events and hospitalization. New England Journal of Medicine. 2004, 351 (13): 1296-1305. 10.1056/NEJMoa041031.View ArticlePubMedGoogle Scholar
- Tonelli M, Wiebe N, Culleton B, House A, Rabbat C, Fok M, McAlister F, Garg AX: Chronic kidney disease and mortality risk: a systematic review. J Am Soc Nephrol. 2006, 17 (7): 2034-2045. 10.1681/ASN.2005101085.View ArticlePubMedGoogle Scholar
- National Collaborating Centre for Chronic Conditions: Chronic kidney disease: national clinical guideline for early identification and management in adults in primary and secondary care. 2008, London: NICE, [http://www.nice.org.uk/Guidance/CG73/Guidance/pdf/English]Google Scholar
- Stevens PE, O'donoghue DJ, de Lusignan S, Van Vlymen J, Klebe B, Middleton R, Hague N, New J, Farmer CK: Chronic kidney disease management in the United Kingdom: NEOERICA project results. Kidney International. 2007, 72 (1): 92-9. 10.1038/sj.ki.5002273.View ArticlePubMedGoogle Scholar
- Gallagher H, O'Donaghue D, de Lusignan S, Ed: Chronic Kidney Disease: Frequently Asked Questions. NHS Employers and the General Practitioners Committee of the British Medical Association, [http://www.nhsemployers.org/SiteCollectionDocuments/Chronic_Kidney_Disease_FAQs_cd_110209.pdf]
- de Lusignan S, Mimnagh C: Breaking the first law of informatics: the Quality and Outcomes Framework (QOF)in the dock. Inform Prim Care. 2006, 14 (3): 153-6. [http://www.ingentaconnect.com/content/rmp/ipc/2006/00000014/00000003/art00001]PubMedGoogle Scholar
- NHS Information Centre: The Quality and Outcomes Framework. [http://www.ic.nhs.uk/statistics-and-data-collections/audits-and-performance/the-quality-and-outcomes-framework]
- Jami P, Smith P, Moningi S, Moningi V, Martin SA, Rosencrance G, Reyes BJ: Compliance with Joint National Committee 7 guidelines in hypertension management in a teaching institution. Am J Med Qual. 2007, 22 (4): 251-8. 10.1177/1062860607303293.View ArticlePubMedGoogle Scholar
- New JP, Middleton RJ, Klebe B, Farmer CK, de Lusignan S, Stevens PE, O'Donoghue DJ: Assessing the prevalence, monitoring and management of chronic kidney disease in patients with diabetes compared with those without diabetes in general practice. Diabet Med. 2007, 24 (4): 364-9. 10.1111/j.1464-5491.2007.02075.x.View ArticlePubMedGoogle Scholar
- de Lusignan S, Teasdale S, Little D, Zapp J, Zuckerman A, Bates DW, Steele A: Comprehensive computerised primary care records are an essential component of any national health information strategy: report from an international consensus conference. Inform Prim Care. 2004, 12 (4): 255-264.PubMedGoogle Scholar
- de Lusignan S, Chan T: The development of primary care information technology in the United kingdom. J Ambul Care Manage. 2008, 31 (3): 201-10.View ArticlePubMedGoogle Scholar
- Ashworth M, Jenkins M, Burgess K, Keynes H, Wallace M, Roberts D, Majeed A: Which general practices have higher list inflation? An exploratory study. Family Practice. 2005, 22 (5): 529-31. 10.1093/fampra/cmi057.View ArticlePubMedGoogle Scholar
- de Lusignan S, van Weel C: The use of routinely collected computer data for research in primary care: opportunities and challenges. Family Practice. 2006, 23 (2): 253-63. 10.1093/fampra/cmi106.View ArticlePubMedGoogle Scholar
- de Lusignan S, Metsemakers JF, Houwink P, Gunnarsdottir V, Lei van der J: Routinely collected general practice data: goldmines for research? A report of the European Federation for Medical Informatics Primary Care Informatics Working Group (EFMI PCIWG) from MIE Maastricht, The Netherlands. Informatics in Primary Care. 2006, 14 (3): 203-9.PubMedGoogle Scholar
- Gomez G, de Lusignan S, Gallagher H: Chronic Kidney Disease: a new priority for primary care. Br J Gen Pract. 2006, 56 (533): 908-910.PubMedPubMed CentralGoogle Scholar
- Eccles MP, Grimshaw JM: Selecting, presenting and delivering clinical guidelines: are there any 'magic bullets'?. Medical Journal of Australia. 2004, 180 (6 Supplement): S52-4.PubMedGoogle Scholar
- Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, Whitty P, Eccles MP, Matowe L, Shirran L, Wensing M, Dijkstra R, Donaldson C: Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess. 2004, 8 (6): iii-iv.View ArticlePubMedGoogle Scholar
- de Lusignan S: An educational intervention, involving feedback of routinely collected computer data, to improve cardiovascular disease management in UK primary care. Methods Inf Med. 2007, 46 (1): 57-62.PubMedGoogle Scholar
- de Lusignan S, Belsey J, Hague N, Dhoul N, van Vlymen J: Audit-based education to reduce suboptimal management of cholesterol in primary care: a before and after study. Journal of Public Health. 2006, 28 (4): 361-369. 10.1093/pubmed/fdl052.View ArticlePubMedGoogle Scholar
- de Lusignan S, Dzregah B, Hague N, van Vlymen J: Using routinely collected general practice data to improve the quality of chronic disease management, for health service planning, and research. In Ed. Bryant J. Current Perspectives in Healthcare Computing 2005: conference proceedings HC2005. 2005, Swindon, BCS HIC, 309-19.Google Scholar
- de Lusignan S, Stephens PN, Adal N, Majeed A: Does feedback improve the quality of computerized medical records in primary care?. Journal of the American Medical Informatics Association. 2002, 9 (4): 395-401. 10.1197/jamia.M1023.View ArticlePubMedPubMed CentralGoogle Scholar
- National Kidney Foundation: Kidney Disease Outcomes Quality Initiative (KDOQI). KDOQI Clinical Practice Guidelines for Chronic Kidney Disease: Evaluation, Classification, and Stratification. Part 4. Definition and classification of stages of chronic kidney disease. Guideline 1. Definition and stages of chronic kidney disease. [http://www.kidney.org/Professionals/kdoqi/guidelines_ckd/p4_class_g1.htm]
- NICE: Chronic kidney disease. Quick Reference Guide. Chronic Kidney Disease Early identification and management of chronic kidney disease in adults in primary and secondary care. CG73. [http://www.nice.org.uk/Guidance/CG73/QuickRefGuide/pdf/English]
- Leipzig RM, Cumming RG, Tinetti ME: Drugs and falls in older people: a systematic review and meta-analysis: II. Cardiac and analgesic drugs. J Am Geriatr Soc. 1999, 47 (1): 40-50.View ArticlePubMedGoogle Scholar
- Gillespie LD, Gillespie WJ, Robertson MC, Lamb SE, Cumming RG, Rowe BH: Interventions for preventing falls in elderly people. Cochrane Database Syst Rev. 2003, CD000340-4
- Klebe B, Irving J, Stevens PE, O'Donoghue DJ, de Lusignan S, Cooley R, Hobbs H, Lamb EJ, John I, Middleton R, New J, Farmer CK: The cost of implementing UK guidelines for the management of chronic kidney disease. Nephrol Dial Transplant. 2007, 22 (9): 2504-12. 10.1093/ndt/gfm248.View ArticlePubMedGoogle Scholar
- Joint Speciality Committee for Renal disease of the Royal College of Physicians of London and the Renal Association: Chronic kidney disease in adults: UK guidelines for identification, management and referral. 2006, Royal College of Physicians of London, [http://www.rcplondon.ac.uk/pubs/contents/54942fe5-ef23-4dc7-aeb8-ed60354ffb23.pdf]Google Scholar
- Ash JS, Berg M, Coiera E: Some unintended consequences of information technology in health care: the nature of patient care information system-related errors. J Am Med Inform Assoc. 2004, 11 (2): 104-12. 10.1197/jamia.M1471.View ArticlePubMedPubMed CentralGoogle Scholar
- Fung CH, Lim YW, Mattke S, Damberg C, Shekelle PG: Systematic review: the evidence that publishing patient care performance data improves quality of care. Ann Intern Med. 2008, 148 (2): 111-23.View ArticlePubMedGoogle Scholar
- Werner RM, Asch DA: The unintended consequences of publicly reporting quality information. JAMA. 2005, 293 (10): 1239-44. 10.1001/jama.293.10.1239.View ArticlePubMedGoogle Scholar
- Marshall MN, Romano PS, Davies HT: How do we maximize the impact of the public reporting of quality of care?. Int J Qual Health Care. 2004, 16 (Suppl 1): i57-63. 10.1093/intqhc/mzh013.View ArticlePubMedGoogle Scholar
- Campbell MK, Elbourne DR, Altman DG: CONSORT group. CONSORT statement: extension to cluster randomised trials. British Medical Journal. 2004, 328 (7441): 702-708. 10.1136/bmj.328.7441.702.View ArticlePubMedPubMed CentralGoogle Scholar
- Department of Health: Confidentiality – NHS Code of practice. 2003, [http://www.dh.gov.uk/prod_consum_dh/groups/dh_digitalassets/@dh/@en/documents/digitalasset/dh_4069254.pdf]Google Scholar
- PRIMIS (Primary Care Information Services): Nottingham University, [http://www.primis.nhs.uk/pages/default.asp]
- de Lusignan S, Hague N, van Vlymen J, Kumarapeli P: Routinely-collected general practice data are complex, but with systematic processing can be used for quality improvement and research. Inform Prim Care. 2006, 14 (1): 59-66.PubMedGoogle Scholar
- van Vlymen J, de Lusignan S, Hague N, Chan T, Dzregah B: Ensuring the Quality of Aggregated General Practice Data: Lessons from the Primary Care Data Quality Programme (PCDQ). Stud Health Technol Inform. 2005, 116: 1010-1015.PubMedGoogle Scholar
- van Vlymen J, de Lusignan S: A system of metadata to control the process of query, aggregating, cleaning and analysing large datasets of primary care data. Inform Prim Care. 2005, 13 (4): 281-291.PubMedGoogle Scholar
- Glynn LG, Buckley B, Reddan D, Newell J, Hinde J, Dinneen SF, Murphy AW: Multimorbidity and risk among patients with established cardiovascular disease: a cohort study. Br J Gen Pract. 2008, 58 (552): 488-94. 10.3399/bjgp08X319459.View ArticlePubMedPubMed CentralGoogle Scholar
- Glynn LG, Reddan D, Newell J, Hinde J, Buckley B, Murphy AW: Chronic kidney disease and mortality and morbidity among patients with established cardiovascular disease: a West of Ireland community-based cohort study. Nephrol Dial Transplant. 2007, 22 (9): 2586-94. 10.1093/ndt/gfm222.View ArticlePubMedGoogle Scholar
- Bowling A: Research methods in health. 2002, Buckingham; Open University Press, 257-309. 2Google Scholar
- Ritchie J, Lewis J, Eds: Qualitative Research Practice. 2003, London, Sage
- Brunel University: Health Economics Research Group (HERG). [http://www.brunel.ac.uk/about/acad/herg/]
- Kerry SM, Bland JM: Analysis of a trial randomised in clusters. British Medical Journal. 1998, 316 (7124): 54-View ArticlePubMedPubMed CentralGoogle Scholar
- Eldridge SM, Ashby D, Kerry S: Sample size for cluster randomized trials: effect of coefficient of variation of cluster size and analysis method. International Journal of Epidemiology. 2006, 35 (5): 1292-300. 10.1093/ije/dyl129.View ArticlePubMedGoogle Scholar
- NHS Information Centre: NHS Workforce: Consultants and GPs as of 31st december 2004. [http://www.ic.nhs.uk/statistics-and-data-collections/workforce/nhs-staff-numbers/nhs-workforce:-consultants-and-gps-as-at-31-december-2004]
- NHS Information Centre: GP Workload survey. [http://www.ic.nhs.uk/pubs/gpworkload]
- Doyal L: Preserving moral quality in research, audit, and quality improvement. Qual Saf Health Care. 2004, 13 (1): 11-2. 10.1136/qshc.2003.008680.View ArticlePubMedPubMed CentralGoogle Scholar
- de Lusignan S, Belsey J, Hague N, Dzregah B: End digit preference in BP recordings of patients with ischaemic heart disease in primary care. Journal of Human Hypertension. 2004, 18: 261-5. 10.1038/sj.jhh.1001663.View ArticlePubMedGoogle Scholar
- de Lusignan S, Wells S, Hague N, Thiru K: Managers see the problems associated with coding clinical data as a technical issue whilst clinicians also see cultural barriers. Methods Inf Med. 2003, 42 (4): 416-22.PubMedGoogle Scholar
- NHS Employers, British Medical Association: Quality and Outcomes Framework. Guidance for GMS Contract 2009/10. Delivering investment in general practice. 2009, London; NHS Confederation (Employers) Ltd, 101-6. [http://www.nhsemployers.org/SiteCollectionDocuments/QUALITY_OUT_COMPLETE_CD_110209.pdf]Google Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.