Skip to main content

Archived Comments for: Do physician outcome judgments and judgment biases contribute to inappropriate use of treatments? Study protocol

Back to article

  1. How do general probability forecasts enter into decision making?

    Paul Falzer, VA Connecticut Healthcare System

    3 July 2009

    This commentary is prompted by two concerns: First, what do we actually know about the role of forecasting in making treatment decisions, and to what extent does the proposal by Brehaut and colleagues shed light on this relationship? If I understand their line of reasoning correctly, they are saying the following: Clinicians often depend on forecasts of outcome to make treatment decisions. These forecasts are in the form of probability statements. Clinicians make forecasts in probabilistic terms by invoking cognitive heuristics, and these can lead to errors in judgment. Hence, and quoting from page 3: “One of the goals of the current work is to determine the extent to which cognitive heuristics such as availability contribute to inappropriate use of treatments by physicians.”

    Their rationale is predicated on the claim that appears on page 2: “There is considerable evidence showing that physicians have trouble accurately judging the probability of important clinical events and outcomes in a variety of clinical settings.” Unfortunately, this claim was supported by a single reference to a one-page convention poster summary, and the summary itself contains no references. It may appear that the statement is almost self-evident, given the wealth of studies about flawed and biased clinical decision making. However, the extant literature presents a more complicated picture of how probability forecasts enter into decisions. Particularly noteworthy are studies by Gigerenzer and associates (1, 2) proponents of naturalistic decision making (3-5), and naturalistic applications in the clinical decision making literature (6-8).

    According to Weiner (9), the central question for clinicians is not, “what is the likelihood in general that a given treatment will produce a good result,” but rather -- and in his words -- “What is the best next thing for this patient at this time?” Weiner says that efforts to assess clinical judgment by adhering to the large of large numbers produce a pattern of sub-optimality known as “context error.” This bias is an exemplar of the fallacy of division -- the proposition that what is true for the whole is true for all of the parts within it. Given the current state of knowledge, I would like to see a clear and well documented line of reasoning in support of the notion that treatment decisions are flawed because they are based on general probability forecasts. In the alternative, perhaps in addition, further discussion may be warranted about the possible influence of the law of large numbers and context errors on implementation research.


    1. Hertwig R, Gigerenzer G. The "conjunction fallacy" revisited: How intelligent inferences look like reasoning errors. J Behav Dec Making 1999;12(4):275-305.

    2. Sedlmeier P, Gigerenzer G. Intuitions about sample size: The empirical law of large numbers. J Behav Dec Making 1997;10(1):33-51.

    3. Beach LR. Epistemic strategies: Causal thinking in expert and nonexpert judgment. In: Wright G, Bolger F, editors. Expertise and decision support. New York: Plenum Press; 1992. p. 107-127.

    4. Beach LR, Barnes VE, Christensen-Szalanski JJJ. Beyond heuristics and biases: A contingency model of judgemental forecasting. Journal of Forecasting 1986;5(3):143.

    5. Klein GA. The fiction of optimization. In: Gigerenzer G, Selten R, editors. Bounded Rationality. Cambridge, MA: MIT Press; 2001. p. 103-112.

    6. Falzer PR, Garman DM, Moore BA, Rohrbaugh RA, Beach LR. Strategies of Evidence-Based Decision Making. Second Annual NIH Conference on the Science of Dissemination and Implementation: Building Research Capacity to Bridge the Gap from Science to Service; Bethesda, MD; 2009.

    7. Falzer PR, Garman DM, Moore BA. Examining the influence of clinician decision making on adherence to a clinical guideline. Psychiatr Serv 2009;60(5):698-701.

    8. Arocha JF, Wang D, Patel VL. Identifying reasoning strategies in medical decision making: A methodological guide. J Biomed Inform 2005;38(2):154-171.

    9. Weiner SJ. Contextualizing medical decisions to individualize care: Lessons from the qualitative sciences. J Gen Intern Med 2004;19(3):281-5.

    Competing interests