Skip to main content

Table 3 Summary of the measures used by included studies, methods of analysis and results of comparisons

From: Are there valid proxy measures of clinical behaviour? a systematic review

Study Proxy measure Direct Measure (DM) Analysis
  Description
1. Method
V = Clinical vignette (No. of case simulations)
CI/Q = Clinician interview/questionnaire
MR = Medical Record review
PI/Q = Patient interview/questionnaire
2. Timing
Clinician self report (SR) Medical Record Review (MR) Patient report (PR) Description
1. Method
SP = Simulated Patients
DO = Direct Observation
VR = Video recording
AR = Audio recording
2. Timing
SP Training reported Psychometrics (IRR) Compared Item by Item Compared Summary Scores Agreement between measures:
Co-efficient r; kappa (k); Structural equation modelling (SEM); Sensitivity (Sens) & Specificity (Spec)
Difference between mean scores:
ANOVA; T-test
P
Stange[5]
1998
1. MR; PQ
2. At end of consultation
  DO   0.39 to 1.00 (kappa)   MR
Sens = 8% (diet advice) – 92% (Lab tests)
Spec = 83% (social history) – 100% (counselling services, physical exam, lab tests)
k = 0.12 to 0.92 (79 comparisons)
PR
Sens = 17% (mammogram) – 89% (Pap test)
Spec = 85% (in-office referral) – 99% (immunisation, physical exam, lab tests)
k = 0.03 to 0.86 (53 comparisons)
NR
Flocke[6]
2004
1. PQ
2. At end of consultation (24%) or postal return (76%)
   DO   NR   Sens* = 11% (substance use) – 76% (smoking cessation) NA
Wilson[7]
1994
1. MR; PQ
2. At end of consultation
  AR   0.79 to 1.00   MR
Sens = 31%, Spec* = 99%
28.6 (Alcohol)
Sens = 29%, Spec* = 100%
83.3 (BP)
Sens = 83%, Spec* = 93%
% agreement between DM & MR:
45.5 (Smoking)
PR
Sens = 74%, Spec* = 94%
75.0 (Alcohol)
Sens = 75%, Spec* = 94%
100 (BP)
Sens = 100%, Spec* = 90%
% agreement between DM & PR:
81.8 (Smoking)
NA
Ward[8]
1996
1. PQ
2. Questionnaire mailed to patient within 2 days of consultation
   AR   0.74 to 0.94 (kappa)   Sens = 93% (smoking status)
Spec = 79%
Sens = 92% (cessation advice)
Spec = 82%
NA
Zuckerman[9]
1975
1. MR
2. At end of consultation
   AR   NR   Sens* = 0% (side effects) – 100% (Diagnosis)
Spec* = 9% (Diagnosis) – 100% (side effects)
NA
Luck[10]
2000
1. MR
2. At end of consultation
   SP (27) each role-playing 1 of 8 case simulations NR ANOVA (4-way)
Necessary care:
Sens = 70%, Spec = 81%
Unnecessary care:
Sens = 65%' Spec = 64%.
<0.0001
NA
Page[11]
1980
1. V (4)
2. Upto 6 weeks before or 3 weeks after SP visit
   SP (4) each role-playing 1 case simulation 0.76 r = .56 & .68
r = .26 & .37
"Must do" actions
Sens* = 97%, Spec* = 33%
"Must not do" actions
Sens* = 30%, Spec* = 98%
>0.05
<0.05
Gerbert[12]
1988
1. CI; MR; PI
2. At end of consultation
√R   NR   k = 0.67 (SR)
k = 0.54 (MR)
k = 0.50 (PR)
<0.001
<0.001
<0.001
Pbert[13]
1999
1. CI; PI
2. At end of consultation
  AR.   NR r = 0.77 (SR)
r = 0.67 (PR)
<0.0001
<0.0001
Gerbert[14]
1986
1. CI; MR; PI
2. At end of consultation
√R   0.52 to 0.93 (kappa)   Median % agreement (All categories):
0.84 (SR)
0.88 (MR)
0.86 (PR)
NA
Dresselhaus[15]
2000
1.V (8); MR
2. NR
  SP (4) each role-playing a simple and complex case presentation NA   ANOVA (3-way) <0.01
Rethans[16]
1987
1. V (1).
2. Completed 2 months after SP visit
   SP (3) each role-playing same case simulation 0.78 to 1.0 (kappa) T-test:
Overall
"Obligatory"
"Intermediate"
"Superfluous"
ns
<0.005
<0.05
<0.05
Rethans[17]
1994
1. MR
2. Charts reviewed two years after SP visit.
   SP (4) each role-playing 1 of 4 case simulations 0.93 (kappa) r = 0.54 (Overall)
r = 0.17 (History taking)
r = 0.45 (Physical exam)
r = 0.75 (Lab exam)
r = 0.50 (Advice)
r = 0.43 (Medication)
r = -0.04 (Follow-up)
<0.05)
ns
ns
<0.01
<0.05
ns
ns
Peabody[18]
2000
1. V (8); MR
2. Completed "several weeks" after SP visit
  SP (4) each role-playing a simple and complex case presentation NA   ANOVA (4-way) <0.001
O'Boyle[19]
2001
1. % time practiced hand hygiene
2. Up to one month prior to observation period
   DO
Nurses observed for 2 hours or until 10 indications for handwashing had occurred
  0.94 to 0.98   r = 0.21
SEM = 0.201
<0.05
<0.05
  1. * Calculated by authors NA = Not applicable NR = Not reported ns = non-significant