Skip to main content

Process evaluation of a complex intervention to optimize quality of prescribing in nursing homes (COME-ON study)



The COME-ON study was a cluster-controlled trial of a complex intervention that consisted of a blended training program, local interdisciplinary meetings, and interdisciplinary case conferences in Belgian nursing homes. The intervention was associated with significant improvements in the appropriateness of prescribing. The aims of this study were to describe the implementation of the intervention and to explore the experiences of participants, for the purpose of identifying factors associated with implementation and perceived impact and to draw lessons for future implementation.


We performed a mixed-method process evaluation. Questionnaires and reports were used to collect quantitative data on implementation and experiences from the 24 NHs and participating healthcare professionals (coordinating physicians, general practitioners, pharmacists, and nurses) in the intervention group. Multidisciplinary focus groups focusing on factors associated with implementation and perceived impact were conducted in 11 NHs.


Overall, the rate of implementation and the satisfaction of participants were good, despite some variability between NHs and HCPs. Although perceived impact on nursing home residents varied, most participants perceived a positive impact for themselves. Factors associated with implementation and perceived impact were identified at different levels: intervention, healthcare professionals, organization, and external context. The interdisciplinary and face-to-face approaches were recognized as key elements for the success of the intervention, despite organizational constraints. The attitude of general practitioners was identified both as a barrier to and a facilitator for implementation and its success. The professional role and competency of the pharmacist influenced perceived impact. The pre-existing relationships between HCPs and the presence of a leader facilitated implementation and perceived impact. Remuneration was deemed necessary for the study and for future implementation.


Overall, the intervention, and more specifically its interdisciplinary aspect, was well implemented and appreciated by HCPs. This probably contributed to the positive effect on the appropriateness of prescribing. Future implementation must take into account the various factors found to affect implementation and perceived impact, in order to maximize effect and sustainability.

Trial registration

Current Controlled Trials ISRCTN66138978; registered 18 November 2015, retrospectively registered,

Peer Review reports


For many years, concerns have been raised about the high prevalence of polypharmacy and potentially inappropriate prescribing (PIP) among nursing home residents (NHRs) [1,2,3]. The presence of PIP has been associated with adverse drug reactions [4, 5], increased healthcare utilization [6,7,8,9,10], and even death [6,7,8, 11]. Reviews of studies on improving the appropriateness of prescribing in the nursing home (NH) setting have produced mixed results [12,13,14,15,16,17,18]. Complex interventions, i.e., interventions with multiple interacting components, are likely to be required [12,13,14,15]. However, there is a need for further high-quality trials to identify which interventions are most effective in which context [12,13,14,15,16,17,18].

Process evaluation is an essential part of designing and testing complex interventions [19, 20]. Besides complexity related to multiple interacting components, complexity can also be due to the difficulty of implementation and the number of organizational levels targeted [21]. Process evaluation within a randomized controlled trial (RCT) of a complex intervention makes it possible to interpret findings and to inform policymakers planning future implementation [22]. Moreover, studies assessing interventions that were context-dependent have lacked process information on, e.g., the level of implementation or the influencing contextual factors; this has made interpretation in systematic reviews difficult [14, 19, 23]. The NH environment and context can vary from one country, region, or NH to another and this can substantially influence the development and effect of approaches to improving appropriateness of prescribing. Process evaluations of RCTs on appropriate prescribing in the NH setting have been scarce [24].

The COME-ON (Collaborative approach to Optimize MEdication use for Older people in Nursing homes) study was a cluster-controlled trial in 54 Belgian NHs to evaluate the impact of a complex intervention on the appropriateness of prescribing [25]. The core element of the intervention was a structured and repeated interdisciplinary review of residents’ medication (referred to hereafter as “interdisciplinary case conferences” (ICCs)) supported by a blended training program and local interdisciplinary meetings (LIMs; referred as to “local concertation” in the protocol paper). General practitioners (GPs), coordinating physicians, nurses, and pharmacists were involved in all components of the intervention. Details about each component of the intervention are provided in Table 1. The protocol has been published previously [26].

Table 1 COME-ON intervention components

The primary outcome measure related to the appropriateness of prescribing at resident level and was considered successful when at least one potentially inappropriate medication (PIM) or potential prescribing omission (PPO) present at baseline had been solved at the end of study and when there were no new PIMs or PPOs at the end of study compared with baseline. Explicit tools were used and incorporated in an algorithm to detect PIMs and PPOs. Using a three-level mixed-effects model accounting for data clustering, a significant effect in favor of the intervention was observed (odds ratio 1.479 [95%CI 1.062–2.059, p = 0.021]; number needed to treat 19). Analysis of PIMs at a criterion level showed slightly better results for the intervention group for some criteria (e.g., criteria related to the use of PPIs or benzodiazepines). Underuse of vitamin D substantially decreased in the intervention group [27].

With regard to medication use data, the median number of medications did not change over time in either group. However, significant decreases in the consumption of antithrombotic agents, drugs for peptic ulcer and gastroesophageal reflux disease, hypnotics, and sedatives were observed in the intervention group but not in the control group. The use of lipid-modifying agents decreased in both groups, but much more in the intervention group. In the intervention group, a significant increase was seen in the use of vitamin A/D. Most clinical outcomes measured (including hospital admission, visit to the emergency department, consultation with GP or specialist physician) did not significantly differ between groups. Data on clinical outcomes must however be interpreted with great caution for different reasons, and these are discussed in detail elsewhere [27].

The aim of the present paper is to describe the process evaluation of the COME-ON study. Specifically, the objectives were to understand how the intervention was implemented and to explore the experiences of participants, for the purpose of identifying factors associated with implementation and perceived impact and to draw lessons for future implementation.



Each Belgian NH is legally bound to have a coordinating physician—a GP with specific additional training—who is, among other things, responsible for promoting rational medication use within the NH through the development of a therapeutic formulary. NHRs have the right to choose their GPs and the number of visiting GPs is unrestricted. The role of pharmacists is mainly limited to the delivery of medications. Standardized medication reviews are not recommended or funded as in Australia or the United States (U.S.A.).

Study design

We followed the Medical Research Council guidance on process evaluation of complex interventions [19]. The three domains of evaluation recommended by the guidance were explored, namely implementation, causal mechanisms, and contextual factors [19]. The “implementation” domain is the exploration of which elements of the intervention are actually delivered (dose), how delivery is achieved (fidelity and adaptations), and whether the intended target group comes into contact with the intervention (reach). The “mechanisms of impact” exploration aims to identify the process through which the intervention produces changes. The third domain involves identifying context elements that positively or negatively affect implementation and outcomes.

Data were collected using a mixed-methods approach combining quantitative and qualitative data. The data collection methods and main aspects evaluated are summarized in Table 2.

Table 2 Data collected for process evaluation

Quantitative data collection

Quantitative data were collected from the 24 intervention NHs.

For the training component, participation in an e-learning course was registered automatically through an e-learning platform and participation in the interdisciplinary workshops was registered using attendance forms. An online satisfaction survey was sent by email to healthcare professionals (HCPs) who took part in the training 3 months after the start of the training period. Two reminders were sent. In designing the survey, we focused on the first two levels of the Kirkpatrick model: reaction (to what degree participants react favorably to the training) and learning (to what degree participants acquire the intended knowledge, skills, attitudes, confidence, and commitment as a result of their participation) [28]. We formulated 25 statements and HCPs indicated their levels of agreement using a 4-point Likert scale (ranging from “strongly disagree” to “strongly agree”). The questionnaire was pilot-tested by three professionals with expertise in this area to check clarity, relevance, and completeness.

For LIMs, implementation data were collected through reports filled in by the coordinating physician and/or head nurse after each meeting and sent by email to the research team. Data collected included general information (topic discussed, number of participants, duration, etc.), level of consensus reached, and conclusions.

For ICCs, implementation data were collected during or straight after each ICC. The HCPs recorded the following information via a dedicated web application: participants, duration, preparation, drug-related problems identified, action taken, and follow-up.

Qualitative data collection

Two months after the end of the study, qualitative data were collected using multidisciplinary focus groups (FGs) in a sample of intervention NHs. Purposive sampling was used to ensure variation in NH characteristics (size, ownership status, and location), the number of included NHRs per GP, previous experience with any form of ICCs, and the research team’s perception of implementation and success, based on regular contact between the research team and NHs. Twelve of the 24 intervention NHs were invited to participate. One contact person per NH was asked to invite eight to ten participants in total and at least one participant in each profession. One NH refused to participate due to lack of time.

Our aim was to collect information about how participants experienced the COME-ON study. In particular, the objectives were to understand in-depth how the intervention was implemented and how it produced changes and to identify barriers and facilitators affecting implementation and perceived impact. The results on the primary outcome were not available when the FGs met. Therefore, FG participants were invited to reflect on their perception of the determinants of the intervention’s success or failure. A semi-structured interview guide was developed and addressed each component of the intervention and each domain of the process evaluation. Participants were also asked to make recommendations for consideration if the intervention were to be scaled up across NHs in Belgium. The guide was tested during the first FG. Only minor modifications were made subsequently and, therefore, the first FG was retained in the final set for analysis. Each FG took place in a quiet room in the NH. One researcher moderated the discussion and a second was present as an observer and took field notes. The discussions were conducted in Dutch for the five NHs in Flanders and in French for the six NHs in Wallonia. Additional focus groups were not conducted because of the saturation of data. The mean duration was 106 minutes and on average eight HCPs were present. The coordinating physician, the pharmacist, and at least one nurse attended all FGs. For three FGs, no GP was present.

Data analysis

Quantitative data were summarized using descriptive statistics using R software (Free Software Foundation, Inc., Boston, MA, U.S.A.).

All FGs were audio-recorded and transcribed verbatim in the original language. Participants were pseudo-anonymized by the assignment of an ID code. NVivo 10 was used to assist with the analysis. Researchers were experienced in qualitative analysis and their background (pharmacists and psychologist) differed so as to enable richer data interpretation.

A thematic analysis was performed using the pre-defined categories based on the themes in the interview guide. The first focus group was coded separately by three researchers (PA, MVG, SR; MVG and SR were external to the main trial team); MVG’s native language was Dutch but she had sufficient command of French to participate in the coding exercise for the first FG (held in French). Agreements were reached on the development of the coding tree after discussion among the three researchers and other members of the research team. Then, two researchers (MVG, SR) coded all transcripts for the focus groups held in their native languages. Consistency of coding was ensured through the use of a common coding book and through frequent meetings among researchers to discuss coding and, when needed, to edit the coding instructions. Afterward, the main barriers and facilitators that influenced the implementation and/or perceived impact of the intervention were classified using the four-level framework defined by Lau et al. [29]. The four key elements that influence the implementation of change include external contextual factors, organization-related factors, individual professionals, and characteristics of the intervention [29]. Researcher bias was minimized through regular cross-checking of data and findings by the members of the research team. We used quotations as exemplars of key themes.

Ethical considerations

The study was approved by the Ethics Committee (s57145, ML11035) and by the Belgian Privacy Commission (SCSZ/14/084/174). All NHRs or residents’ representatives provided written informed consent. FG participants provided oral consent for audio-recording.

Study registration

This study has been registered at (trial registration number: ISRCTN66138978).


Blended training

In total, 71% (268/378) of all HCPs in the intervention group participated in the training. Participation rates varied according to profession, with GPs having the lowest participation rate. Full implementation data are presented in Table 3.

Table 3 Implementation data for each component of the COME-ON intervention

One hundred and twenty-nine HCPs answered the satisfaction survey (response rate 48%, 129/268). Overall, the blended training was highly appreciated. With regard to the e-learning, 90% of respondents found that the content met their personal needs and 80% found its level of difficulty reasonable. Ninety-four percent self-reported an improvement in knowledge about appropriate medication use in older people and 98% in knowledge about conducting an interdisciplinary medication review. Ninety-four percent saw the content of the interdisciplinary workshops as relevant to COME-ON activities. In general, participants felt comfortable about discussing cases with colleagues, but fewer nurses (78%) and pharmacists (70%) than physicians (95%) felt comfortable about discussing the appropriate use of medicines with other professions. The interdisciplinary aspect was highly appreciated: 92% of respondents agreed that the workshop had stimulated collaboration with other HCPs within their NH. Furthermore, most respondents indicated an improvement in their confidence in actively participating in ICCs (84%) and in the skills required for performing a medication review (88%). The combination of the e-learning platform and the interdisciplinary workshops was considered “complementary” by 91% of respondents. Detailed results of the satisfaction survey are presented in Additional file 1.

Overall, data from the FGs confirmed the results of the satisfaction survey. Although the flexibility of the e-learning was appreciated by some HCPs, others—mainly nurses—found it difficult to find time to do it during working hours and showed reluctance to do it at home. The interdisciplinary workshop was preferred by some HCPs because of its interactivity and its planned/scheduled aspect, as well as by those lacking computer skills.

Local interdisciplinary meetings

The majority of the NHs (22/24) carried out two LIMs. Two NHs organized only one meeting because of planning difficulties, even when the first meeting was appreciated, or unwillingness, because no benefit had been perceived after the first LIM. Participation rate per profession varied from 57% for GPs to 100% for coordinating physicians. An interdisciplinary team (i.e., at least one GP, one pharmacist, and one nurse) was present at most meetings (83%, 38/46). In two NHs, no GP attended either LIMs. In one of these, the coordinating physician did not dare to invite GPs because he considered that the workload due to the study was too high. On the other hand, he believed the notions discussed during the LIM should already be familiar to the other GP. Full implementation data are presented in Table 3.

A large majority of NHs reported having achieved a partial or complete consensus on the use of both medication classes for which support material was provided, i.e., antidepressants (91%, 21/23) and lipid-lowering drugs (95%, 20/21). However, data from the FGs show that opinions were divided on the level of consensus and the perceived impact of the LIM. Consequently, perception of the value of scaling up LIM meetings in Belgian NHs differed.

Some participants stated that the meetings did not have an impact because no real consensus was reached due to the lack of constructive discussion and/or the low level of GPs’ attendance. Even when a consensus was reached, some HCPs did not perceive an impact and/or were not sure that it led to changes in prescribing habits. The reasons cited included GPs were not convinced by the principle of selecting one or two first-choice drug(s), GPs clearly stated that they were attached to their therapeutic freedom, and the goal of the study was not always understood (e.g., fear of control). Other reasons included the feeling that no improvement was needed and the observation that the LIM was too late in the study to have an impact on the ICC. Participants in some NHs, however, perceived a positive impact: discussion of a specific medication class made it possible to bear the latest evidence in mind and made it easy to apply what they had agreed to the participating NHRs during the ICCs and also more broadly to all NHRs.

In spite of these mixed feelings on perceived impact, the LIMs seem to have been generally appreciated, especially the training aspect and the exchange of experiences. Nurses, in particular, but also some GPs, recognized that they learned something about the medication classes discussed. Moreover, with this extra knowledge, nurses reported feeling more confident about asking GPs questions about medications for individual NHRs.

The main barriers and facilitators that influenced the implementation and perceived impact of the LIM are presented in Table 4.

Table 4 Factors that influenced the implementation and/or the perceived impact of local interdisciplinary meetings

Interdisciplinary case conferences

A total of 1675 ICCs were registered for 681 NHRs. The median number of ICCs per NHR was 3 (P25-P75, 2–3; range, 1–4). The proportion of NHRs with at least three ICCs varied greatly between NHs, from 0% to 100% (median 84%, [P25-P75] 54-94). Full implementation data are presented in Tables 3 and Additional file 2.

For 90% of ICCs, at least one nurse, one pharmacist, and one GP were present. The median duration of ICC meetings was 15 minutes.

The median number of drug-related problems (DRPs) recorded per NHR was 4.8, but varied from 0 to 8 in different NHs. The median number of DRPs per NHR was 2 for the first ICC and 1 for the second and third ICCs (Additional file 2). FG participants saw the first ICC as the most important, as many DRPs were discussed there. The second and the third ICC were seen as useful follow-ups by some HCPs and as unnecessary by others.

Overall, participants were satisfied with the ICCs. They appreciated time being taken to have an interdisciplinary patient-centered discussion about medications, under better conditions than the short exchanges that occur during routine GP visits. The study raised their awareness of the need to conduct regular medication reviews. However, the perceived impact of the ICCs on NHRs varied greatly. Some HCPs mentioned that few treatment changes were implemented, either because they considered them unnecessary or due to inertia or reluctance to change. Moreover, they stated that it was more difficult to change a medication prescribed by a specialist physician or medications for NHRs with psychiatric disorders. Some HCPs were unable to identify whether or not the ICC had an impact. Others reported an impact in a reduction of the number of medications and sometimes a cost reduction. Improvements in NHRs’ conditions and/or quality of life were seldom reported.

Besides perceptions of the impact on NHRs, participating HCPs recognized some benefits for themselves. Some nurses reported being more confident about asking GPs questions about medications and being increasingly involved in the resident’s follow-up. Some pharmacists stated that, thanks to the study, other HCPs started to acknowledge their role. Moreover, the relationships between the HCPs involved improved throughout the study.

Globally, residents and relatives were not informed or only briefly informed about decisions taken at ICCs. The presence of NHRs would have been perceived as a barrier; although most participants mentioned that it is important to involve residents and/or relatives to some extent, they did not know how exactly this should be done.

The main barriers and facilitators affecting the implementation and impact of ICCs are presented in Table 5.

Table 5 Factors that influenced the implementation and/or the perceived impact of the interdisciplinary case conferences (ICCs)

First, the main intervention-related facilitators were the interdisciplinary and face-to-face approaches, which contributed to the success of ICCs, despite the organizational constraints. Likewise, the preparation made, mainly by pharmacists, was considered an advantage. Second, at the level of professionals, we found variability in expectations regarding the roles of other HCPs. This was particularly true of the role of the pharmacist, whose contribution some found positive and others confronting. A positive attitude by GPs was acknowledged as a facilitator when present and its lack was seen as a barrier. Third, the key facilitator at the organizational level was the presence of a leader who promoted/stimulated the implementation of the intervention. Preexisting positive relations between the HCPs involved also facilitated the intervention. Finally, HCPs were paid for their participation in the COME-ON study. This external contextual factor was appreciated and could be seen as a facilitator.

For future implementation, FG participants raised concerns about feasibility, mainly due to time and resource constraints and a lack of motivation among non-participating GPs. Funding would have to be provided and they suggested reducing the number of ICCs per resident from three to one or maximum two per year and stated that hospital discharge could offer a good opportunity to perform a medication review, even if this had not been performed during the study.


This process evaluation used a mixed-methods approach in order to understand to what extent the COME-ON intervention was implemented as foreseen and to explore participants’ experiences. It made it possible to identify barriers and facilitators, to reflect on how implementation may have influenced outcomes and to highlight lessons for future implementation. Overall, implementation and satisfaction were good, which probably contributed to the positive effect seen on the primary outcome measure, but variation was observed between NHs and HCPs. Several elements might explain the variability in implementation and perceived impact. These are discussed below.


The training component was highly appreciated—including its interprofessional aspect—and was perceived as an important step before LIM and ICCs. The online training and the face-to-face workshops were considered complementary and we believe that both approaches should be offered in the future. Even though there is little current evidence of the impact of educational approaches on the quality of prescribing [14, 18], it is often considered an essential component of complex interventions. Moreover, in a recent study, GPs’ knowledge of geriatric pharmacotherapy improved after a short e-learning module [30]. It is also noteworthy to comment that the medication classes on which an effect was seen in the intervention group (e.g., proton pump inhibitors, vitamin D) were all included in the “short list of frequent PIMs and PPOs”. This list was a core component of the training provided. Several participants of the focus groups highlighted the usefulness of this list, which is reflected in these data. The innovative aspects of the COME-ON training were the combination of e-learning and onsite training and its interprofessional approach. As far as we know, this approach had never been tested in the nursing home setting.

Opinions on LIMs diverged. While the training aspect and the exchange of experiences were generally appreciated, divergent views were expressed on the impact on prescribing and NHRs. Many GPs did not fully support the aim of reaching consensus on the drugs to be integrated into a therapeutic formulary, as it was perceived as limiting physicians’ freedom in prescribing. An evolution in attitudes on this matter requires time. The focus group data suggest that the class of medication discussed during LIMs may influence impact, i.e., that for more complex themes (antidepressant was mentioned as more complex that lipid-lowering drugs), the implementation may be lower. This is in line with the findings on medication use data, which show a significant decrease in the use of lipid-lowering drugs, but no major modification in the use of antidepressants. Given the positive experiences reported with LIMs in other countries such as Switzerland [31], the low cost, and complementarity with ICCs, we believe that LIMs—with clear instructions and adequate material—should be maintained and be scaled up in the future. The interprofessional aspect of both training and LIM is key, as our data suggest that this enabled the development of a common language across the team and improved communication processes. This may have a downstream effect on the appropriateness of prescribing [24].

The interdisciplinary and face-to-face approaches of ICCs were identified as facilitators for the success of the intervention, but also as barriers to implementation due to organizational constraints. The face-to-face approach has been acknowledged to contribute to effective teamwork because it fosters the establishment of a trusting relationship [32, 33]. Therefore, for future implementation, it seems important that ICCs are conducted face-to-face. However, as suggested during the FGs, once a trusting relationship has been established, other forms of communication (e.g., teleconsultation) would be potential alternatives. Overall, participants saw the workload and frequency of ICCs as heavy and most were concerned about the feasibility of scaling up ICCs to all NHRs. As in other countries [34], a yearly interdisciplinary medication review for each NHR seems more appropriate.


Consistently with the literature, the presence of the GP during ICCs was regarded as essential [35]. However, GP participation in the different components of the intervention was a challenge, even though the participant GPs were volunteers. Several factors were identified: organizational constraints, lack of motivation, the goal of the study is unclear, unclear expectations, reluctance to have their therapeutic decisions questioned, and a large number of GPs per NH. These factors have already been mentioned in the literature [36]. In many countries where an open staff model composed of independent physicians is the norm, there is a need for more medical governance in long-term care, in order to improve the accountability of attending physicians [37]. Our results confirm that the role of the coordinating physicians should be strengthened and that NHs must develop a stronger policy on medication use that must be accepted by all attending GPs.

In contrast to studies in other countries such as the U.S.A. and Australia [38], actively involving pharmacists was an innovative step. We observed variations in GPs’ expectations of the role of pharmacists. Given the lack of close collaboration in daily practice with community pharmacists, GPs might not acknowledge their expertise [32]. A clear definition of roles and responsibilities with regard to medication review is required, as it is one of the key determinants of effective interprofessional collaboration [33, 39]. Another common barrier to effective collaboration with GPs is inadequate clinical training of pharmacists [40]. In the present study, some pharmacists stressed their lack of knowledge and skills in medication review as an important barrier. Consistently with the literature, we strongly believe that all HCPs involved could benefit from additional training in geriatric pharmacotherapy [41]. However, if there is a willingness to extend the pharmacists’ role in the NH setting, they certainly need more training. Taking a lead from Australia, we could imagine mandatory training for pharmacists to obtain accreditation for working in the NH setting [34].


There were also factors in NHs as organizations that had a positive influence on the implementation and impact of the intervention, such as preexisting relations between HCPs [42], the presence of a leader, and staff resources. The organizational culture and prescribing culture may also have influenced the implementation and impact [43, 44]. Several studies have suggested that the culture in NHs is linked to variations in the use, for example, of psychotropic medicines [45, 46].

External context

Funding was provided for ICCs and certainly contributed to the good implementation rates. It seems obvious that future implementation of ICCs must be remunerated. The clinical trial context also positively influenced implementation. Particular attention should be paid to real-world implementation data, if any component of our intervention is to be adopted in routine practice.

Strengths and limitations of this study

This study is one of the first studies to have conducted an in-depth pre-planned process evaluation of an intervention implemented in NHs that aimed to improve the quality of prescribing [24, 19]. A number of important findings emerged from the process data; these help us to understand the implementation of the intervention and participant satisfaction and to identify lessons for future implementation. Examples of final recommendations made to Belgian stakeholders at the end of the study and for which process data strongly influenced the recommendation are provided in Additional file 3.

However, this study has some limitations. We used FGs to collect qualitative data. Participatory observation supplemented by individual interviews could have provided an even fuller picture of how different factors influenced implementation and impact, but this was not possible due to time constraints. Due to the multidisciplinary nature of FGs, participants may have been reluctant to mention barriers relative to interprofessional collaboration in front of other HCPs. The FGs were moderated by members of the COME-ON study team, some of whom had regular contact with participating NHs, so some participants may have felt uncomfortable about voicing their views. Moreover, FG participants may also have been more enthusiastic HCPs. However, we observed variations in opinions and experiences between FGs. Therefore, we can reasonably assume that the process evaluation made it possible to identify the main enablers and barriers. We did not explore the views of NHRs or relatives. Finally, we did not quantify associations between the present process data (e.g., level of implementation, HCP or organization-related factors) and the primary outcome measure. Those should be studied in future research.


The process evaluation of the COME-ON study shows that the intervention was, overall, well implemented, including with regard to its interprofessional aspect, and was appreciated by HCPs under certain conditions. Careful development and piloting of the intervention likely contributed to good implementation and satisfaction data, and these in turn probably contributed to the positive effect seen on the appropriateness of prescribing. The current process evaluation data also suggest that a larger effect could be reached with several adaptations such as reinforcing the role and competency of the pharmacist, strengthening interprofessional relationships, and improving medical governance and GP’s accountability. Future implementation must take into account the various factors—at the intervention, professional, organizational, and external context levels—that were found to affect implementation or impact, in order to maximize effect and sustainability.

Availability of data and materials

Part of the data generated or analyzed during this study is included in this article and its supplementary information files or is available from the corresponding author on reasonable request. The full datasets generated or analyzed during the current study are not publically available because consent to make data publically available was not part of the consent by participants.



Collaborative approach to Optimize Medication use for Older people in Nursing homes


Focus group


General practitioner


Healthcare professional


Interdisciplinary case conference


Local interdisciplinary meetings


Nursing home


Nursing home resident


Potentially inappropriate prescribing


Randomized controlled trial


United States of America


  1. 1.

    Morin L, Laroche ML, Texier G, Johnell K. Prevalence of potentially inappropriate medication use in older adults living in nursing homes: a systematic review. J Am Med Dir Assoc. 2016;17(9):862 e1–9.

    Article  Google Scholar 

  2. 2.

    Storms H, Marquet K, Aertgeerts B, Claes N. Prevalence of inappropriate medication use in residential long-term care facilities for the elderly: a systematic review. Eur J Gen Pract. 2017;23(1):69–77.

    Article  Google Scholar 

  3. 3.

    Jokanovic N, Tan EC, Dooley MJ, Kirkpatrick CM, Bell JS. Prevalence and factors associated with polypharmacy in long-term care facilities: a systematic review. J Am Med Dir Assoc. 2015;16(6):535 e1–e12.

    Article  Google Scholar 

  4. 4.

    Hedna K, Hakkarainen KM, Gyllensten H, Jonsson AK, Petzold M, Hagg S. Potentially inappropriate prescribing and adverse drug reactions in the elderly: a population-based study. Eur J Clin Pharmacol. 2015.

  5. 5.

    Hamilton H, Gallagher P, Ryan C, Byrne S, O’Mahony D. Potentially Inappropriate medications defined by STOPP criteria and the risk of adverse drug events in older hospitalized patients. Arch Internal Med. 2011;171(11):1013–9.

    Article  Google Scholar 

  6. 6.

    Perri M 3rd, Menon AM, Deshpande AD, Shinde SB, Jiang R, Cooper JW, et al. Adverse outcomes associated with inappropriate drug use in nursing homes. Annals Pharmacother. 2005;39(3):405–11.

    Article  Google Scholar 

  7. 7.

    Wauters M, Elseviers M, Vaes B, Degryse J, Dalleur O, Vander Stichele R, et al. Too many, too few, or too unsafe? Impact of inappropriate prescribing on mortality, and hospitalization in a cohort of community-dwelling oldest old. Br J Clin Pharmacol. 2016.

  8. 8.

    Lau DT, Kasper JD, Potter DE, Lyles A, Bennett RG. Hospitalization and death associated with potentially inappropriate medication prescriptions among elderly nursing home residents. Arch Internal Med. 2005;165(1):68–74.

    Article  Google Scholar 

  9. 9.

    Dalleur O, Spinewine A, Henrard S, Losseau C, Speybroeck N, Boland B. Inappropriate prescribing and related hospital admissions in frail older persons according to the STOPP and START criteria. Drugs Aging. 2012;29(10):829–37.

    Article  Google Scholar 

  10. 10.

    Moriarty F, Bennett K, Cahir C, Kenny RA, Fahey T. Potentially inappropriate prescribing according to STOPP and START and adverse outcomes in community-dwelling older people: a prospective cohort study. Br J Clin Pharmacol. 2016.

  11. 11.

    Muhlack DC, Hoppea LK, Weberpalsa J, HB MD, Schöttker B. The association of potentially inappropriate medication at older age with cardiovascular events and overall mortality: a systematic review and meta-analysis of cohort studies. JAMDA. 2017;18:211–20.

    PubMed  Google Scholar 

  12. 12.

    da Silva DT, Santos AP, Aguiar PM, da Silva WB, de Lyra DP, Jr. Analysis of research quality regarding pharmaceutical intervention in elderly residents of long-term care facilities: a systematic review. J Am Geriatr Soc. 2010;58(7):1404–6.

    Article  Google Scholar 

  13. 13.

    Loganathan M, Singh S, Franklin BD, Bottle A, Majeed A. Interventions to optimise prescribing in care homes: systematic review. Age Ageing. 2011;40(2):150–62.

    Article  Google Scholar 

  14. 14.

    Forsetlund L, Eike MC, Gjerberg E, Vist GE. Effect of interventions to reduce potentially inappropriate use of drugs in nursing homes: a systematic review of randomised controlled trials. BMC Geriatr. 2011;11:16.

    Article  Google Scholar 

  15. 15.

    Kaur S, Mitchell G, Vitetta L, Roberts MS. Interventions that can reduce inappropriate prescribing in the elderly: a systematic review. Drugs Aging. 2009;26(12):1013–28.

    Article  Google Scholar 

  16. 16.

    Marcum ZA, Handler SM, Wright R, Hanlon JT. Interventions to improve suboptimal prescribing in nursing homes: A narrative review. Am J Geriatr Pharmacother. 2010;8(3):183–200.

    Article  Google Scholar 

  17. 17.

    Verrue CL, Petrovic M, Mehuys E, Remon JP, Vander SR. Pharmacists' interventions for optimization of medication use in nursing homes: a systematic review. Drugs Aging. 2009;26(1):37–49.

    Article  Google Scholar 

  18. 18.

    Alldred DP, Kennedy MC, Hughes C, Chen TF, Miller P. Interventions to optimise prescribing for older people in care homes (Review). Cochrane Collab. 2016.

  19. 19.

    Moore G, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions (UK Medical Research Council (MRC) guidance). 2015.

  20. 20.

    Coly A, Parry G. Evaluating Complex health interventions: a guide to rigorous research designs. Academy Health. 2017.

  21. 21.

    Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M, et al. Developing and evaluating complex interventions: the new Medical Research Council guidance. Bmj. 2008;337:a1655.

    Article  Google Scholar 

  22. 22.

    Moore G, Audrey S, Barker M, Bond L, Bonell C, Cooper C, et al. Process evaluation in complex public health intervention studies: the need for guidance. J Epidemiol Commun Health. 2013;68(2):101–2.

    Article  Google Scholar 

  23. 23.

    Rankin A, Cadogan CA, Patterson SM, Kerse N, Cardwell CR, Bradley MC, et al. Interventions to improve the appropriate use of polypharmacy for older people. Cochrane Database Syst Rev. 2018;9:CD008165.

    PubMed  Google Scholar 

  24. 24.

    Desveaux L, Saragosa M, Rogers J, Bevan L, Loshak H, Moser A, et al. Improving the appropriateness of antipsychotic prescribing in nursing homes: a mixed-methods process evaluation of an academic detailing intervention. Implement Sci. 2017;12(1):71.

    CAS  Article  Google Scholar 

  25. 25.

    American Geriatrics Society Beers Criteria Update Expert P. American Geriatrics Society 2015 Updated Beers Criteria for Potentially Inappropriate Medication Use in Older Adults. J Am Geriatr Soc. 63(11):2015, 2227–4.

  26. 26.

    Anrys P, Strauven G, Boland B, Dalleur O, Declercq A, Degryse J-M, et al. Collaborative approach to Optimise MEdication use for Older people in Nursing homes (COME-ON): study protocol of a cluster controlled trial. Implement Sci. 2016;11(1).

  27. 27.

    Strauven G, Anrys P, Vandael E, Henrard S, Lepeleire JD, Spinewine A, et al. Cluster-Controlled trial of an intervention to improve prescribing in nursing homes (COME-ON study). J Am Med Dir Assoc. in press;2019.

  28. 28.

    Kirkpatrick. Evaluating training programs: The four levels. 2nd ed ed. San Francisco: Berrett-Koehler Publishers; 1998.

  29. 29.

    Lau R, Stevenson F, Ong BN, Dziedzic K, Treweek S, Eldridge S, et al. Achieving change in primary care--causes of the evidence to practice gap: systematic reviews of reviews. Implement Sci. 2016;11:40.

    Article  Google Scholar 

  30. 30.

    Shane C, Denis OM, Stephen B. Use of an e-learning educational module to better equip doctors to prescribe for older patients: a randomised controlled trial. Drugs Aging. 2017;34(5):367–74.

    Article  Google Scholar 

  31. 31.

    Locca JF, Ruggli M, Buchmann M, Huguenin J, Bugnon O. Development of pharmaceutical care services in nursing homes: practice and research in a Swiss canton. Pharm World Sci. 2009;31(2):165–73.

    Article  Google Scholar 

  32. 32.

    Chen TF, de Almeida Neto AC. Exploring elements of interprofessional collaboration between pharmacists and physicians in medication review. Pharm World Sci. 2007;29(6):574–6.

    Article  Google Scholar 

  33. 33.

    Levenson SA, Saffel DA. The consultant pharmacist and the physician in the nursing home: roles, relationships, and a recipe for success. J Am Med Dir Assoc. 2007;8(1):55–64.

    Article  Google Scholar 

  34. 34.

    Sluggett JK, Ilomaki J, Seaman KL, Corlis M, Bell JS. Medication management policy, practice and research in Australian residential aged care: Current and future directions. Pharmacol Res. 2017;116:20–8.

    Article  Google Scholar 

  35. 35.

    Nazir A, Unroe K, Tegeler M, Khan B, Azar J, Boustani M. Systematic review of interdisciplinary interventions in nursing homes. J Am Med Dir Assoc. 2013;14(7):471–8.

    Article  Google Scholar 

  36. 36.

    Kjell H. Halvorsen PSaAGG. A qualitative study of physicians' and nurses' experiences of multidisciplinary collaboration with pharmacists participating at case conferences. Int J Pharmacy Practice. 2011;19:350–7.

    Article  Google Scholar 

  37. 37.

    Smith DA, Guest K. Is education all that's needed to reduce inappropriate drugs in long-term care? J Am Med Dir Assoc. 2015;16(1):9–11.

    Article  Google Scholar 

  38. 38.

    Hughes CM, Roughead E, Kerse N. Improving use of medicines for older people in long-term care: contrasting the policy approach of four countries. Healthc Policy. 2008;3(3):e154–67.

    PubMed  PubMed Central  Google Scholar 

  39. 39.

    Mion L, Odegard PS, Resnick B, Segal-Galan F. Geriatrics Interdisciplinary Advisory Group AGS. Interdisciplinary care for older adults with complex needs: American Geriatrics Society position statement. J Am Geriatr Soc. 2006;54(5):849–52.

    Article  Google Scholar 

  40. 40.

    Kwint H-F, Bermingham L, Faber A, Gussekloo J, Bouvy ML. The Relationship between the extent of collaboration of general practitioners and pharmacists and the implementation of recommendations arising from medication review. A systematic review. Drugs Aging. 2013;30:91–102.

    Article  Google Scholar 

  41. 41.

    Cullinan S, Fleming A, O'Mahony D, Ryan C, O'Sullivan D, Gallagher P, et al. Doctors' perspectives on the barriers to appropriate prescribing in older hospitalized patients: a qualitative study. Br J Clin Pharmacol. 2015;79(5):860–9.

    Article  Google Scholar 

  42. 42.

    Bradley F, Ashcroft DM, Noyce PR. Integration and differentiation: a conceptual model of general practitioner and community pharmacist collaboration. Res Social Adm Pharm. 2012;8(1):36–46.

    Article  Google Scholar 

  43. 43.

    Hughes CM, Lapane K, Watson MC, Davies HT. Does organisational culture influence prescribing in care homes for older people? A new direction for research. Drugs Aging. 2007;24(2):81–93.

    Article  Google Scholar 

  44. 44.

    Tjia J, Gurwitz JH, Briesacher BA. Challenge of changing nursing home prescribing culture. Am J Geriatr Pharmacother. 2012;10(1):37–46.

    Article  Google Scholar 

  45. 45.

    Zuidema SU, de Jonghe JF, Verhey FR, Koopmans RT. Psychotropic drug prescription in nursing home patients with dementia: influence of environmental correlates and staff distress on physicians' prescription behavior. Int Psychogeriatrics/IPA. 2011;23(10):1632–9.

    Article  Google Scholar 

  46. 46.

    Chen Y, Briesacher BA, Field TS, Tjia J, Lau DT, Gurwitz JH. Unexplained variation across US nursing homes in antipsychotic prescribing rates. Arch Internal Med. 2010;170(1):89–95.

    Article  Google Scholar 

Download references


We would like to thank Françoise Docq, Jean-Marie Degryse, and Muriel Rocour for their help with the evaluation of satisfaction of the blended training. We are most grateful to Robert Houdart for his contribution to the development of the web application and his technical support. We gratefully acknowledge the nursing homes, directory board, healthcare professionals, and residents that participated in the study. We also thank all members of the consortium for their contribution to the COME-ON study: Benoit Boland, Olivia Dalleur, Anja Declercq, Jean-Marie Degryse, Valérie Lacour, Steven Simoens, Niko Speybroeck, and Kris Vanhaecht.


This study was funded by the National Institute for Health and Disability Insurance (NIHDI) (Belgium). The funding body had a limited role in the design, but had no role in data collection, implementation, analysis, or the decision to publish the results. PA and GS were paid by the NIHDI.

Author information




PA, GS, VF, and AS contributed to the conception of the mixed methods evaluation and developed the survey questionnaire and interview guides. PA conducted the data collection and initial analysis relative to the training questionnaire. PA, GS, JDL, AS, and VF conducted the focus groups. SR, MV, and PA conducted the initial analysis relative to the focus groups. All authors contributed to the analysis as part of the project research group. PA wrote the initial draft of the manuscript. AS and VF provided the critical review and editing of the initial and subsequent drafts of the manuscript. All authors critically revised the manuscript, read, and approved the final manuscript.

Corresponding authors

Correspondence to Veerle Foulon or Anne Spinewine.

Ethics declarations

Ethics approval and consent to participate

The COME-ON trial was approved by the Ethics Committee of UZ Leuven on November 12, 2014 (reference number s57145, ML11035). Furthermore, this study was also approved by the Belgian Privacy Commission on October 21, 2014 (Beraadslaging nr. 14/095, reference: SCSZ/14/084/174). All included NHRs or residents’ representatives provided their written informed consent before the beginning of the study. The director, coordinating physician, chief nurse, and pharmacist also provided written informed consent before the beginning of the study. Completion of the training questionnaire implied consent to participate; written information was provided at the start of the online questionnaire about the purpose of the questionnaire, the voluntary nature of participation and assurances that no individual data would be disclosed. Healthcare professionals gave oral consent for participation and audio-recording before starting the focus groups.

Consent for publication

Not applicable

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Anrys, P., Strauven, G., Roussel, S. et al. Process evaluation of a complex intervention to optimize quality of prescribing in nursing homes (COME-ON study). Implementation Sci 14, 104 (2019).

Download citation


  • Potentially inappropriate prescribing
  • Process evaluation
  • Complex intervention
  • Mixed methods
  • Nursing homes