In a vignette-based study with 168 emergency physicians, we used signal detection theory to quantify the perceptual sensitivity and decisional threshold of physicians making trauma triage decisions. We found that cognitive processes in physician decision-making may contribute to persistent rates of under-triage in trauma triage. When responding to the cases, many physicians in the study had low perceptual sensitivity, making triage decisions only weakly related (if at all) to the ACS-COT standard. Additionally, most had a high decisional threshold for transfer, systematically erring on the side of not transferring patients to regional trauma centers.
Many epidemiological studies have shown that patients with moderate to severe injuries are routinely under-triaged [8–11]. However, those studies have typically focused on patient-level determinants of variability in triage decisions. Chang et al. have described age as a determinant of triage decisions . Macias et al. posit that age, co-morbidities, and severity of injury influence the triage of patients with traumatic spinal cord injuries to trauma centers . We used signal detection theory, a basic decision science method, to determine how physician cognitive processes might influence rates of under-triage. Although rarely used for this purpose, signal detection theory has the advantage that it distinguishes between sensory and decisional components of decision making. For example, by using signal detection theory to analyze nursing risk assessments for patients admitted to hospital, Thompson et al. have shown that time–pressure and clinical experience influence variability in decision making through different mechanisms. We hypothesized that signal detection theory would allow us to identify reasons for non-compliance with clinical practice guidelines. Better understanding of the cognitive processes responsible for under-triage would provide valuable information for the design of future quality improvement interventions .
We presented physicians with a series of case vignettes that required a range of triage decision-making. We found that when answering the case vignettes most physicians demonstrated limited perceptual sensitivity, as shown in Figure 4. The ACS-COT uses ATLS, an educational program that operationalizes the clinical guidelines, as one of its primary tools to standardize the treatment of trauma patients. However, although most physicians in our sample had received ATLS certification, the regression weight on the guidelines in our models suggested that triage decisions on this questionnaire corresponded only weakly with the ACS-COT criteria for the transfer of patients.
Patterns in the responses suggested potential explanations for the limited perceptual sensitivity demonstrated by these physicians. For example, as shown in Table 3, among one-half of the under-triaged patients, physicians correctly identified the patient as meeting the standard for transfer. However, they delayed transport to obtain additional diagnostic imaging. Acquisition of this imaging suggested triage decision-making consistent with the disjunction effect. As described by Shafir and Tversky, people have difficulty making decisions in the context of complex uncertainty. One manifestation of that difficulty is the pursuit of non-instrumental information, which appears relevant but if available would not impact decision making [23, 24]. In effect, the information from imaging acquired for patients with moderate to severe injuries would have had negative value, as it reduced the chances of successful treatment.
Similarly, the greater likelihood of transferring patients with penetrating (rather than blunt) injury suggested reliance on the representativeness heuristic. Formally equivalent problems should provoke equivalent judgments. In other words, people should estimate the probability of x belonging to set A in the same way that they estimate the probability of y belonging to set A. However, Kahneman and Tversky have shown that when people rely on the representativeness heuristic to make judgments, they substitute the perceived similarity of x or y to other objects in set A for that probability estimate . In this context, cases with blunt injuries seemed less likely to need tertiary center care as otherwise equivalent cases with penetrating injuries. This hypothesis would also explain patterns of patient-level variability in triage. For example, if physicians judge severe trauma as the purview of young men, they may systematically under-triage elderly patients, as described by Chang et al. and Macias et al. [8, 10].
Figure 4 also demonstrates that the group of physicians who responded to the questionnaire had a high threshold for transfer. To target physicians’ decisional thresholds, the ACS-COT has relied on trauma systems, voluntary networks of local community hospitals that associate with high-volume trauma centers . Through outreach and accreditation programs, it has urged physicians at non-trauma centers to have a low decisional threshold in order to minimize under-triage. In other words, it advocates transferring as many patients as necessary to ensure the capture of all those with moderate to severe injuries, even at the cost of increasing over-triage . Despite these efforts, our sample of physicians preferred to err on the side of minimizing over-triage. Free-text comments by participants suggested that their decisional thresholds reflected conscious negative attitudes towards transferring patients, invoking issues such as resentment towards guidelines in general and distaste at relinquishing control over patient care. The finding of lower decisional thresholds among physicians at hospitals with an established trauma center affiliation suggested that organizational norms or incentives may play roles as well. For example, hospitals without these affiliations may encourage their physicians to avoid transferring patients out of network to prevent the loss of revenue.
Current quality improvement efforts in trauma assume that the same barriers to compliance with clinical practice guidelines affect all physicians equally. However, the individual performance differences revealed in Figure 5 suggest the need for a more nuanced approach. For example, some physicians had a high threshold for transferring patients, and an above average ability to discriminate between patients with minor and moderate-severe injuries. These doctors seemed to at least partially compensate for their apparent unwillingness to transfer patients through skill at distinguishing between those who really did ‘need’ transfer and those who did not. Other physicians had a threshold for transferring patients set around zero and a lower than average perceptual sensitivity. These doctors seemed to at least partially compensate for their apparent inability to distinguish between those who did and did not ‘need’ transfer by being more willing to transfer everyone.
Physicians with decisional thresholds biased away from transfer and with above-average perceptual sensitivity would most likely benefit from an intervention that recalibrated their decisional threshold, perhaps by addressing their concerns about transferring patients or creating financial incentives for the appropriate transfer of patients. In contrast, exposing the second type of physician with a decisional threshold already set around zero to that same intervention might have unwanted consequences. Specifically, increasing their willingness to transfer patients would increase over-triage, and could impose a burden on level I centers. An analysis of triage patterns in Pennsylvania by Mohan et al. demonstrates that simply shifting decisional thresholds to achieve ACS-COT targets for triage would result in a five-fold increase in transfers to trauma centers . Moreover, non-trauma centers would lose an important source of revenue and the opportunity to provide care for patients in their community. Instead, these physicians might benefit from a strategy that modified their heuristics, perhaps through training with stimuli like the vignettes used in this study .
Our study had several limitations. First, we used vignettes to measure physicians’ perceptual sensitivity and decisional threshold, which did not replicate the time or organizational pressures of clinical decision-making, and with less information than may be available in real-life. Evidence suggesting that case vignettes can predict physician practice patterns comes primarily from the outpatient setting. For example, Peabody et al. demonstrated that physicians’ management of ‘paper patients’ with common clinical conditions, like lower back pain, corresponded to their management of real patients [27, 28]. Our specific vignettes have the content validity that comes from basing them on the case histories of patients treated at the University of Pittsburgh Medical Center. Additionally, the vignettes were extensively pretested. However, we have no knowledge of whether physician decision-making in response to a case vignette-based questionnaire corresponds to actual practice patterns. An alternative approach, and a possibility for future research, might be using an administrative dataset to calculate physicians’ perceptual sensitivity and decisional threshold based on actual triage decisions.
Second, we used the ACS-COT guidelines as our reference standard for transfer, despite flaws that might affect acceptance by physicians: all the stakeholders did not participate in their design ; the definition of over-triage creates an association between prevalence of injury and performance , and the benchmarks may lack feasibility . Yet, robust observational data supports their overall validity. MacKenzie et al. among others, have shown that patients meeting ACS-COT criteria for transfer have better outcomes when treated at trauma centers . Moreover, their widespread dissemination makes them the de facto standard for decision making among physicians, regardless of their potential failings.
Third, we used a non-representative sample of cases on our questionnaire to allow the use of signal detection theory. We did systematically vary the complexity of the cases to elicit the range of decisions that physicians would perform in practice. However, we used a much higher proportion of cases with moderate to severe injuries than physicians would see in practice. We speculate that the absence of non-trauma cases made the triage task easier than the one faced by physicians in their EDs. However, the higher base rate of severe injuries may have altered customary practice patterns. Birnbaum has shown that the predicted effect of the base rate on signal detection estimates depends on the theory of judgment used . We therefore have no clear feeling for how this bias would have affected the patterns observed here. As a test of possible learning effects and base rate influence on the parameter estimates, we compared performance on the first half of the study relative to the second half and found no differences.
Finally, we recruited physicians at a national meeting of emergency medicine physicians, which might limit the generalizability of our observations. However, because physicians attending academic meetings likely have greater knowledge of current clinical practice guidelines, we assume that any bias introduced by our sampling frame would be in the direction of better practice.