Table 3 Summary of the measures used by included studies, methods of analysis and results of comparisons
From: Are there valid proxy measures of clinical behaviour? a systematic review
Study | Proxy measure | Direct Measure (DM) | Analysis | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Â | Description 1. Method V = Clinical vignette (No. of case simulations) CI/Q = Clinician interview/questionnaire MR = Medical Record review PI/Q = Patient interview/questionnaire 2. Timing | Clinician self report (SR) | Medical Record Review (MR) | Patient report (PR) | Description 1. Method SP = Simulated Patients DO = Direct Observation VR = Video recording AR = Audio recording 2. Timing | SP Training reported | Psychometrics (IRR) | Compared Item by Item | Compared Summary Scores | Agreement between measures: Co-efficient r; kappa (k); Structural equation modelling (SEM); Sensitivity (Sens) & Specificity (Spec) Difference between mean scores: ANOVA; T-test | P |
Stange[5] 1998 | 1. MR; PQ 2. At end of consultation |  | √ | √ | DO |  | 0.39 to 1.00 (kappa) | √ |  | MR Sens = 8% (diet advice) – 92% (Lab tests) Spec = 83% (social history) – 100% (counselling services, physical exam, lab tests) k = 0.12 to 0.92 (79 comparisons) PR Sens = 17% (mammogram) – 89% (Pap test) Spec = 85% (in-office referral) – 99% (immunisation, physical exam, lab tests) k = 0.03 to 0.86 (53 comparisons) | NR |
Flocke[6] 2004 | 1. PQ 2. At end of consultation (24%) or postal return (76%) |  |  | √ | DO |  | NR | √ |  | Sens* = 11% (substance use) – 76% (smoking cessation) | NA |
Wilson[7] 1994 | 1. MR; PQ 2. At end of consultation |  | √ | √ | AR |  | 0.79 to 1.00 | √ |  | MR Sens = 31%, Spec* = 99% 28.6 (Alcohol) Sens = 29%, Spec* = 100% 83.3 (BP) Sens = 83%, Spec* = 93% % agreement between DM & MR: 45.5 (Smoking) PR Sens = 74%, Spec* = 94% 75.0 (Alcohol) Sens = 75%, Spec* = 94% 100 (BP) Sens = 100%, Spec* = 90% % agreement between DM & PR: 81.8 (Smoking) | NA |
Ward[8] 1996 | 1. PQ 2. Questionnaire mailed to patient within 2 days of consultation |  |  | √ | AR |  | 0.74 to 0.94 (kappa) | √ |  | Sens = 93% (smoking status) Spec = 79% Sens = 92% (cessation advice) Spec = 82% | NA |
Zuckerman[9] 1975 | 1. MR 2. At end of consultation |  | √ |  | AR |  | NR | √ |  | Sens* = 0% (side effects) – 100% (Diagnosis) Spec* = 9% (Diagnosis) – 100% (side effects) | NA |
Luck[10] 2000 | 1. MR 2. At end of consultation |  | √ |  | SP (27) each role-playing 1 of 8 case simulations | √ | NR | √ | √ | ANOVA (4-way) Necessary care: Sens = 70%, Spec = 81% Unnecessary care: Sens = 65%' Spec = 64%. | <0.0001 NA |
Page[11] 1980 | 1. V (4) 2. Upto 6 weeks before or 3 weeks after SP visit | √ |  |  | SP (4) each role-playing 1 case simulation | √ | 0.76 | √ | √ | r = .56 & .68 r = .26 & .37 "Must do" actions Sens* = 97%, Spec* = 33% "Must not do" actions Sens* = 30%, Spec* = 98% | >0.05 <0.05 |
Gerbert[12] 1988 | 1. CI; MR; PI 2. At end of consultation | √ | √ | √ | √R |  | NR | √ |  | k = 0.67 (SR) k = 0.54 (MR) k = 0.50 (PR) | <0.001 <0.001 <0.001 |
Pbert[13] 1999 | 1. CI; PI 2. At end of consultation | √ |  | √ | AR. |  | NR | √ | √ | r = 0.77 (SR) r = 0.67 (PR) | <0.0001 <0.0001 |
Gerbert[14] 1986 | 1. CI; MR; PI 2. At end of consultation | √ | √ | √ | √R |  | 0.52 to 0.93 (kappa) | √ |  | Median % agreement (All categories): 0.84 (SR) 0.88 (MR) 0.86 (PR) | NA |
Dresselhaus[15] 2000 | 1.V (8); MR 2. NR | √ | √ |  | SP (4) each role-playing a simple and complex case presentation | √ | NA | √ |  | ANOVA (3-way) | <0.01 |
Rethans[16] 1987 | 1. V (1). 2. Completed 2 months after SP visit | √ |  |  | SP (3) each role-playing same case simulation | √ | 0.78 to 1.0 (kappa) | √ | √ | T-test: Overall "Obligatory" "Intermediate" "Superfluous" | ns <0.005 <0.05 <0.05 |
Rethans[17] 1994 | 1. MR 2. Charts reviewed two years after SP visit. |  | √ |  | SP (4) each role-playing 1 of 4 case simulations | √ | 0.93 (kappa) | √ | √ | r = 0.54 (Overall) r = 0.17 (History taking) r = 0.45 (Physical exam) r = 0.75 (Lab exam) r = 0.50 (Advice) r = 0.43 (Medication) r = -0.04 (Follow-up) | <0.05) ns ns <0.01 <0.05 ns ns |
Peabody[18] 2000 | 1. V (8); MR 2. Completed "several weeks" after SP visit | √ | √ |  | SP (4) each role-playing a simple and complex case presentation | √ | NA |  | √ | ANOVA (4-way) | <0.001 |
O'Boyle[19] 2001 | 1. % time practiced hand hygiene 2. Up to one month prior to observation period | √ |  |  | DO Nurses observed for 2 hours or until 10 indications for handwashing had occurred |  | 0.94 to 0.98 |  | √ | r = 0.21 SEM = 0.201 | <0.05 <0.05 |