Why study audit and feedback?
A&F is a convenient term for a heterogeneous group of interventions centered around providing feedback on existing practice to healthcare providers. A&F interventions involve the development of a summary of some aspect of clinical performance (audit) over a specific period of time, and subsequent provision of that summary (feedback) to individual practitioners, teams, or healthcare organisations. A&F has been shown to be effective in a wide variety of clinical contexts, and is one of the most commonly employed and evaluated KT interventions. However, there is enormous variability; a recent Cochrane review of 140 A&F trials showed highly variable effectiveness, ranging from substantial positive effects to null and even negative effects
. Such variability is at least in part due to lack of understanding of the causal mechanisms underlying A&F interventions.
The extensive A&F literature has recently been the subject of a variety of theory-guided systematic reviews, because more standard meta-analytic subgroup analyses (e.g., group size, number of interventions) rarely shed light on causal mechanisms. The most recent Cochrane review reported that the effectiveness of these interventions depends in important ways on how the feedback is presented, such as the source, frequency, delivery format, and whether there is a specific target and action plan
. Two theory-specific reviews have targeted whether the effectiveness of A&F interventions is related to Feedback Intervention Theory
[26, 27] and Control Theory
[28, 29]. A recent review of the explicit role of theory use in A&F trials shows that relatively few trialists appear to have considered any theory during the development of their interventions
These reviews show immense variability between A&F studies in terms of target audience, intervention details, targeted practice change, and context of the interventions. Without knowledge of the relevant causal mechanisms, one cannot predict whether a successful intervention will generalize, learn much from failed interventions, or successfully optimize future interventions
. As an analogy, studies assessing the effectiveness of new drugs would rarely be successful without considerable foundational work explicating the underlying biological mechanisms. Without similar foundational work, KT interventions such as A&F are likely to continue to be hit-and-miss propositions. In the next section, we argue that theory construction using a menu of constructs approach may have advantages over simple application of existing theories.
Applying theories versus constructing theories using a menu of constructs
Initial work applying theory to better understand KT techniques and processes has been mostly drawn from theories of behavior from health and social psychology. For example, the Theory of Planned Behavior (TPB) describes changes in behaviors (e.g., smoking cessation, changes in antibiotics prescribing habits) as being primarily determined by individuals’ intentions to engage in the behavior. Intention, in turn, is primarily determined by three factors: attitude towards the behavior, subjective norms (what important others think of the behavior), and perceived behavioral control (whether the person feels that the behavior is under their control). TPB is influential in discussions around the use of theory in KT, in part because it can be usefully applied in so many contexts (across 16 different studies of provider behaviors, these constructs correlate strongly to changes in target behavior, on average accounting for 31% of the variability
), in part because it is a relatively simple theory to describe and explore
, and in part because of its ubiquity; until recently, the vast majority of theory-informed efforts to change health behaviors, particular health provider behaviors, involved versions of the TPB
Targeting broad, generalizable theories like the TPB as an initial step towards understanding KT interventions has a lot to recommend it. Such theories focus on real-world, observable behavior as the key construct to be explained, rather than, for example, theories of human memory that are built dominantly on human performance in experimental settings and may be more difficult to generalize to non-laboratory-based settings. Simple constructs such as those comprising the TPB can be reasonably understood without an extensive disciplinary background, important in any interdisciplinary field. While constructs like ‘transfer appropriate processing’ (i.e., the notion that the match between how information is encoded in memory and how it is to be retrieved will influence the likelihood its being remembered)
 may well be relevant to many KT interventions, in its entirety it is a complex concept that is unlikely to be readily unpacked by non-specialists. The TPB specifically also comes with established methodologies for measuring the relevant constructs
, an extremely useful criterion for content experts who may be new to the application of theory in their area.
Despite these advantages, detailed theoretical understanding of KT interventions requires investigation beyond broad theories like TPB. For example, the TPB has been criticized as a theory of KT intervention for being better at explaining intention to engage in the behavior (on average, 59% of intention is explained) than it is at explaining the behavior itself (31%)
, for having relatively little to say about how to change and improve KT interventions that have been found to be ineffective
, and for focusing only on voluntary human behavior, when so much of health practice and behavior has at least some automatic, rather than explicitly intentional, component
To us, these general criticisms suggest a need to ‘drill down’ into specifics, to understand and describe more detailed constructs underlying the contexts, interventions, and behaviors in question. Many such constructs exist within the discipline of cognitive psychology, the scientific discipline devoted to understanding the basic mechanisms underlying human thought, including perception, memory, categorization, and judgment and decision making
. Many cognitive constructs seem to have face validity in the KT context and suggest specific, testable, predictions about how interventions might be made more effective. As such, they should be explored in order to examine their utility for describing and improving KT interventions.
Some work has already begun to explore the practice of combining constructs from different theories. Eccles et al.
 conducted a postal survey of 230 Scottish general practitioners around management of upper respiratory tract infections without antibiotics. Noting the range of health and social psychological theories available and the lack of data on their relative merits, the study examined the extent to which constructs from a range of theories predicted hypothetical vignette-based decisions, and actual clinical behavior. Results showed that the model that explained the most variance involved constructs from multiple theories, as opposed to models restricted to an individual theory. A study looking at oral radiography behavior among 214 Scottish dental practitioners showed similar results (i.e., more variance explained when using constructs from multiple theories than any individual theory on its own
). While Foy
 provides a counterexample, these two studies provide intriguing initial empirical evidence to support our claim that incorporation of constructs from different theories (i.e., what we are calling a menu of constructs approach) may lead to advances in understanding KT interventions.
Examples of cognitive constructs worth exploring
We see A&F interventions as a series of mechanisms designed to improve alignment between a practitioner’s practice, the practitioner’s beliefs about his or her actual practice, and best practices as defined by the broader professional community. While all KT interventions seek to align actual practice with best practice, A&F is one of the few that also explicitly targets the fact that individual practitioners rarely have ready access to accurate information on their practice patterns
. A great deal of work in cognitive and educational psychology may shed light on the most effective mechanisms for enabling this alignment, but remains wholly unexplored in the KT literature. Because of the level of abstraction at which these theories were originally conceived, however, it is unlikely that any one theory will provide a complete picture of how A&F may be optimized, thus creating the need to pick and choose individual constructs from multiple theories to determine how they might apply to specific A&F contexts. Below, we present some examples of constructs that may suggest important causal mechanisms related to A&F. All of these constructs have been extensively studied, but few, if any, have been considered in the context of KT interventions. We can, therefore, offer little empirical data as to their impact on the effectiveness of A&F specifically. Instead our intent is simply to indicate how identifying such constructs can make explicit, testable predictions that can inform future research and development efforts in this applied domain.
Two modes of reasoning
One of the most important theoretical perspectives to come out of cognitive psychology is the notion of two modes of reasoning
[38, 39], generically referred to as dual-processing theory. One mode, System 1
, can process information quickly, intuitively, and with relatively little effort. In medicine, development of knowledge structures that allow complex decisions to be made quickly is considered a cornerstone of medical expertise
[40, 41] and likely accounts for a great many of the decisions made during the course of any health provider’s day. The other mode, System 2, can be characterized as slow, analytic, deliberative, and effortful. Patient cases where a provider must stop to think and problem-solve invoke System 2 processing.
Some implications of this important dichotomy in human reasoning for KT have been outlined elsewhere
[42, 43]. A&F as a KT strategy most often invokes the System 2 mode of reasoning; information about current practice must be interpreted and understood, recommendations suggested by guidelines must be considered, and practice change implemented as deemed appropriate. The extent to which this effortful, deliberative process can affect a practice decision that is governed by System 1 processing is unclear, because the A&F literature has not generally been informed by this theoretical approach; interventions are not designed with these notions in mind, and reports do not describe interventions in these terms. However, in the context of designing A&F interventions, dual-process theory can make explicit recommendations about how to improve interventions including understanding the nature of the processing involved in the target decision, employing multi-factor interventions to target different processing modes, and considering cognitive strategies that take advantage of the strengths of both forms of reasoning
Cognitive dissonance and information discounting
Dual processing suggests that a disconnect may exist between the system targeted by A&F interventions and the system dominating actual decision-making, and that this might help explain why simple provision of feedback via an A&F task is not sufficient to ensure practice change. If this is true, one needs to ask what mechanisms might help determine whether or not the feedback is effective. One lens we can apply to this problem is the notion of cognitive dissonance
[45, 46], the distressing mental state that arises when we find ourselves acting in a way that is inconsistent with something we believe (e.g., prescribing antibiotics, perhaps because one’s patients desire them, while believing their use should be limited) or holding two conflicting beliefs at the same time (e.g., I am a good physician and my patients aren’t receiving best possible care). The state is sufficiently uncomfortable that many studies have suggested we are highly motivated to reduce the conflict experienced. The most common result in these situations is that rather than altering behavior or abandoning a belief altogether, the tension is resolved by the easier act of reinterpreting the beliefs (e.g., I am a good physician and my patients are different so the guidelines don’t apply).
An important attribute of this information discounting stemming from cognitive dissonance is that it often happens without conscious awareness (i.e., it resides in System 1)
. There is a substantial literature
 that suggests that these sorts of automatic reinterpretations of available data are commonplace, preventing us from knowing that we are falling prey to them. Indeed, this kind of process has been proposed to be central to a generalized psychological ‘immune system’
 we all possess that involves automatic adaptive tendencies to rationalize in a way that enables us to maintain a sense of well-being and personal strength. These constructs clarify why, instead of accepting feedback and adjusting behavior accordingly, the result may more often be to discount the feedback itself (e.g., ‘the data are not representative of my practice,’ ‘the data are biased,’ or ‘the data do not come from a credible source’). Failing to account for constructs such as these may prove a barrier to successful A&F intervention. Empowering feedback recipients to determine what data will be most relevant to their practice and how it is collected (i.e., guiding the audit side of A&F) may reduce the likelihood that feedback is discounted.
Feedback and feed forward
Work on the Feedback Intervention Theory, alluded to earlier, suggests further ways in which theoretical constructs can identify barriers that may threaten the effectiveness of an A&F intervention. For example, Feedback Intervention Theory might help explain why contextualizing feedback through displays of how the target provider compares to peers (a common strategy in A&F) can be precisely the wrong thing to do. Studies have shown that drawing attention to the recipient’s self-efficacy (i.e., providing normative cues that prompt one to think specifically about where one stands relative to others along some continuum) can create a threat that makes it less likely that the feedback will have an influence
[26, 50]. Rather than serving as a dispassionate indicator of where improvement is possible, such feedback can invoke the psychological immune system’s defense mechanisms, again yielding cognitive dissonance and leading to the data being discounted rather than altering one’s self-assessments. Indeed, this can happen regardless of the sign of the feedback; relatively poor performance can encourage information discounting for the sake of ego defense, while relatively good performance can lead one to believe any deviations from best practice are minor and, hence, there is little to be gained from continued efforts at improvement.
Presenting data in a manner that does not create conflicting beliefs in the first place (i.e., minimizes cognitive dissonance by not engendering the concern that one’s performance is substandard) may, therefore, provide an important consideration for those designing A&F interventions. Kluger and van Dijk’s
 feed forward strategy offers an intriguing possibility that needs to be tested in an A&F context. It involves interviewing the individual about positive past experiences to help them establish an internal standard of excellence and strengthening memory traces of good performances that will influence later System 1 (similarity-based) processing, rather than focusing on the distance between the individual’s performance and an external standard. Early results suggest the procedure has the potential to enable insights into how performance can be improved without damaging self-efficacy
Desirable difficulties and cognitive load
While concerns about the amount of time and cognitive resources practitioners have available to dedicate to contemplating practice improvement leads many interventionists to design A&F interventions to be as simple and accessible as possible, this approach does not consider the clear gains to be derived from requiring the right kind of effort on the part of the target provider. Bjork has put forward the notion of desirable difficulties
[52–54], which suggests that we are better able to learn, remember, and make use of information when we are put in situations that induce errors. These models are aimed deliberately at helping people overcome the false perception that they have understood and learned material in a way that will enable its use in the future. For example, being tested on material has robustly been shown to yield better learning than studying the same material multiple times, even though study often yields feelings of fluency that we erroneously infer to indicate that learning has taken place
. Testing (i.e., being required to effortfully retrieve information from memory), increases the likelihood that we will be able to retrieve the information from memory again in the future and makes it more likely that our attention will be productively focused on areas of knowledge that require further study
. If implemented well, such desirable difficulties might also improve cognitive dissonance by making it more difficult to discount information that one has exerted effort to collect.
One can further specify this issue in terms of cognitive load, a notion that requires us to consider the various types of load that can impact on our mental processing capacity. Intrinsic load (the amount of information to be learned), extraneous load (created by the way in which the information is presented), and germane load (the resources required of working memory to deal with intrinsic and extraneous load) are believed to play different roles in learning. Too many competing factors (e.g., having to read through pages of text in a busy clinical environment to understand the feedback provided by an audit) create too much extraneous load and suboptimal learning. Presenting material with a degree of difficulty greater than the learner is prepared to process can similarly increase intrinsic load to the point of disadvantage
. By contrast, too little germane load (i.e., not engaging working memory to a great enough extent) can create a situation where feedback recipients passively accept information, but are not convinced sufficiently (or prompted to elaborate on the information enough) for the learning to have a long-term impact. Considerable research, largely focused on designing multimedia learning platforms, has demonstrated principles that can optimize these various kinds of load
[57–59]. Consideration of these principles in light of A&F interventions may be a fruitful area of research.
Communities of practice and adaptive expertise
While the examples used to this point have largely focused on the psychology of the individual learner, A&F interventions are often targeted at teams or practices rather than individuals
[60–65]. The notion of Communities of Practice
 describes that social networks of individuals who share a concern and interact regularly around that topic offer substantial benefits for learning. The three crucial characteristics, according to Wenger, are a clearly identifiable domain (i.e., an area of shared competence that distinguishes members of the community from others), the community itself (with relationships nurtured to support and help one another in the group’s joint activity), and practice (i.e., activity oriented around a shared repertoire of experience, expertise, and resources). Practicing in isolation has been found to be a main predictor of underperformance
. Such Communities of Practice might be thought of as informal, ongoing A&F opportunities, and suggest ways in which relationship-centered education
 and mentoring can be incorporated with opportunities for desirable difficulties and reduced cognitive dissonance to develop novel models of A&F.
These models remind us of theoretical notions of adaptive expertise
[68, 69], which suggest that expertise should not be conceived of as a ‘thing’ that can be achieved, but rather, should be considered an approach to continuous efforts at quality improvement. Engaging in this way requires a reward structure that the respect, nurturing, and collegiality of one’s peers can create. Individuals need to feel safe in discussing their clinical practice, but at the same time, simply conveying information to them with no opportunity to discuss the issues with others and come to some mutual understanding of how to alter performance appropriately increases the likelihood that one may inappropriately discount external data that should not be discounted. At the end of the day, these models, when combined with various political and economic theories, suggest that establishing a system whereby the reward structure for healthcare providers encourages deliberate effort to engage with A&F may prove particularly influential in ensuring their effectiveness.