Table 5 Summary of results of comparative case analysis for six audit and feedback sessions with different physician groups. The program model elements, which formed the framework table used for data analysis and indexing are on the vertical axis of this table and the cases are listed across the horizontal. The results presented below are summarized, for purposes of the publication, from the detailed notations captured in the original framework tables. The data sources from which the data were extracted are described in Table 1
Program model elements | Case 1 | Case 2a | Case 2b | Case 2c | Case 2d | Case 3 | |
|---|---|---|---|---|---|---|---|
Innovation | Review of practice variation and outcomes of anesthesia for procedure x | Review of prescribing practice variation for medications in population x at hospital A | Review of prescribing practice variation for medications in population x at hospital B | Review of prescribing practice variation for medications in population x at hospital C | Review of prescribing practice variation for medications in population x at hospital D | Survey of practice variation in anesthesia for 5 procedures at Hospital A. A survey/overview of practice type of project in preparation for a more subsequent specific project. Intent to introduce the group to A&F | |
For this project, A&F was one component of a multifaceted intervention: It was preceded by a didactic inter-professional (doctors, nurses, pharmacists) continuing medical education session, and subsequently included a communication campaign and changes to physician order entry sets, which stemmed from this series of AGFS (cases 2A–D) | |||||||
Style of audit report | Highly refined with heavy group input | The reports across cases 2A–D presented the same aggregate data, but had individual physician data for the participating physicians at each site. In each of these four AGFS, a significant amount of time was spent with the participants asking questions to try to understand the data as it was presented | Engaged group lead contributed substantially to report design | ||||
One of the two project leads was from this site and helped to refine the question for the AF reports | One of the two project leads for this work came from this site and helped to co-create and refine the question for the AF reports | 2D Participants commented during the AGFS that the report was difficult to interpret during the AGFS | |||||
Gold standard for proposed best practice | None | No, but several existing guidelines and recommendations | No, but existing evidence and guidelines. | ||||
Recipients | Specialists at hospital 1. 18 physician reports, 13 physicians attended feedback sessions | Generalist physician group, nurse managers, QI lead at hospitals A, B, C, and D respectively (64 physician reports, 28 physicians attended feedback sessions) | Specialist group, hospital D 17 physician reports, 9 physicians attended feedback session | ||||
Participants in this group identified that their patient population included hospice patients and felt that this influenced prescribing behaviors | Participants in this group cared for very high volume services with high degrees of acuity and complexity | Participants at this site identified that they care for an older patient population. This site had access to a geriatric psychiatry service which may have influenced prescribing patterns. They were positively oriented, cohesive, non-judgmental | Participants at this site self-identified as being “younger” and felt that their prescribing behaviors were influenced heavily by their training and by the local consultants with whom they collaborated | ||||
Context | Project origin | A second project with CPLP for this group | A question raised by city-wide Innovation Committee who invited CPLP to develop the AF reports and deliver AGFS across the 4 sites | CPLP invited by this group after they learned about case 1 | |||
Group dynamic and leadership involvement for the A&F sessions | Experienced, respected senior physician with strong commitment to professional development. Senior physician and 3 colleagues involved in all aspects of project. Cohesive, collegial group. Senior physician and 1 colleague led session and shared own data. | One project lead was a member of this group but was not able to attend this session. The site lead was present but did not have facilitator role in session. Nonetheless, CPLP staff identified that the site lead was “a strong lead” and contributed to the session meaningfully. Lots of questioning and discussion about why this site might be different vs others, skepticism regarding data, requests for data that were not included in original project. CPLP medical director-led session | Site lead present. Lots of questioning of data and discussion about how the data was analyzed. Site lead expressed concern re potential defensiveness of group prior to AGFS. Good attendance. CPLP medical director-led session | One of the two project leads was from this site. Some good discussion in session, but CPLP staff noted that participants began sharing and comparing data as the CPLP team left the room. Good attendance. Project lead shared own data. CPLP medical director-led session | Young physician group in a new hospital working in 2 week shifts with lots of handover. Poor attendance, few questions at session. CPLP medical director-led session | Very motivated, dynamic site project lead with an admin position for education within the group. A young physician cohort, digitally literate, open to self-reflection, keen to discuss change and understand findings. Group cohesion and collegiality high, in part because of call structure. CPLP medical director-led session. Physician project helped prompt group discussion and led discussion around evidence | |
Pre-work, relationship-building | This was a second project for this group. Lots of interaction with CPLP team in project development | This was a first project for each of the four hospital groups. There were several meetings with the two project leads (From cases 2A and 2C) and a meeting with the city-wide QI group (at site 2B), but not with group members | First contact was by invitation: CPLP presented to the entire physician group to introduce concept of A&F program. Project lead met frequently with MD group and CPLP team | ||||
Co-creation of data/metrics | A working group with a strong lead and three department members and input from MD group developed metrics and report | This was one part of a larger project. Individual physicians at each site did not choose what question would be addressed; however, the question was brought to the CPLP by the two project leads (From sites 2A and 2C) and was aligned with a provincial initiative to de-prescribe in this population. The project leads helped the CPLP team to determine which data points to build into the reports | Strong leadership from site lead and collaborative development of project between site lead and MD group members | ||||
Proximity of doctor to data and patient | Direct physicians administering orders at bedside | In-direct: Admitting physician was not always attending, many orders were PRN and at discretion of nursing staff. MDs did not feel they had good control over nurse discretion nor order entry system | Direct, as in case 1, and MDs had direct access to database designers for data capture/metrics | ||||
Facilitation of Session | Co-led by CPLP and group champion who presented own data to group. Prompting and questions from CPLP and from co-facilitator | CPLP facilitator, who was a physician but not a group member, led each session. Facilitation followed structure of the report, table by table. For case 2C, exemplar data from the site lead’s report was presented but the session was still based on structure of the report. Some degree of input from site lead occurred at sites 2A, B, but not at site C or D. The facilitation style at site D was different from the previous sessions. This session was largely a didactic presentation from the facilitator to the group. This was the last of four sessions in this project. The facilitator asked fewer questions than in other groups. When interviewed, the facilitator described the feeling that this group was ‘not as interested’ in the report and that attendance at this session was poor | Led by CPLP facilitator with substantive input from project lead. Prompting questions from both CPLP facilitator and project lead | ||||
Engagement/change talk/action planning | Very interactive. Lots of sharing of personal practices and planning around additional change activities | Moderate interaction. Some change cues were raised, leading to plans for change, and some sharing of practices occurred | Moderate. A participant spontaneously raised need for change and this led to considerable discussion of how this could be achieved including engagement of nursing staff and making a collective commitment to de-prescribing | Minimal. There was little change discussion during this AGFS, but participants did share and compare practices to some extent | Minimal. The group asked questions related only to understanding the data. No change cues arose in this AGFS | Very dynamic session with lots of questions on evidence directed to the project lead and discussion between group members | |
Other outputs | Led to 7 subsequent projects with same specialty in different settings with different innovations and to development of continuous reporting platform for all doctors in this specialty | This AGFS occurred after the education campaign that was a part of the larger initiative. There was no change in prescribing from baseline to this AGFS. However, 1 year after this AGFS, there was a relative reduction in prescribing over all four sites of 39% during study period. Engagement of group in a follow-up project looking at another innovation around insulin prescribing, also in conjunction with a larger, health-authority initiated educational intervention. Changes in order-sets, engagement of multi-disciplinary teams | Specific feedback in follow-up from one group member who changed opioid practice subsequently after learning about practice variation vs other members. Led to follow-up projects with same group looking at other clinical questions and ultimately additional projects in city-wide department (see case 1) | ||||