Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Conditions for production of interdisciplinary teamwork outcomes in oncology teams: protocol for a realist evaluation

  • Dominique Tremblay1, 2Email author,
  • Nassera Touati3,
  • Danièle Roberge1, 2,
  • Jean-Louis Denis3,
  • Annie Turcotte2 and
  • Benoît Samson1, 2, 4
Implementation Science20149:76

https://doi.org/10.1186/1748-5908-9-76

Received: 1 May 2014

Accepted: 11 June 2014

Published: 17 June 2014

Abstract

Background

Interdisciplinary teamwork (ITW) is designed to promote the active participation of several disciplines in delivering comprehensive cancer care to patients. ITW provides mechanisms to support continuous communication among care providers, optimize professionals’ participation in clinical decision-making within and across disciplines, and foster care coordination along the cancer trajectory. However, ITW mechanisms are not activated optimally by all teams, resulting in a gap between desired outcomes of ITW and actual outcomes observed. The aim of the present study is to identify the conditions underlying outcome production by ITW in local oncology teams.

Methods

This retrospective multiple case study will draw upon realist evaluation principles to explore associations among context, mechanisms and outcomes (CMO). The cases are nine interdisciplinary cancer teams that participated in a previous study evaluating ITW outcomes. Qualitative data sources will be used to construct a picture of CMO associations in each case. For data collection, reflexive focus groups will be held to capture patients’ and professionals’ perspectives on ITW, using the guiding question, ‘What works, for whom, and under what circumstances?’ Intra-case analysis will be used to trace associations between context, ITW mechanisms, and patient outcomes. Inter-case analysis will be used to compare the different cases’ CMO associations for a better understanding of the phenomenon under study.

Discussion

This multiple case study will use realist evaluation principles to draw lessons about how certain contexts are more or less likely to produce particular outcomes. The results will make it possible to target more specifically the actions required to optimize structures and to activate the best mechanisms to meet the needs of cancer patients. This project could also contribute significantly to the development of improved research methods for conducting realist evaluations of complex healthcare interventions. To our knowledge, this study is the first to use CMO associations to improved empirical and theoretical understanding of interdisciplinary teamwork in oncology, and its results could foster more effective implementation in clinical practice.

Keywords

Interdisciplinarity Professional practices Realist evaluation Patient outcomes Case study Cancer

Background

Given the nature of their illness and its multimodal treatment, persons with cancer receive care from multiple professionals from different disciplines, working in a variety of settings, whose services may be provided either concurrently or at different points in time [1, 2]. They are particularly susceptible to problems of fragmented or poorly coordinated care, communication failures, and breakdowns in care continuity, all of which compromise their timely access to needed care [3]. This results in unmet healthcare needs, with potentially harmful repercussions on patients’ health and well-being [4, 5]. To address these deficiencies deplored by persons with cancer and their families, clinicians, and decision-makers, several countries such as Canada, the United Kingdom, the United States, and Australia have made interdisciplinary teamwork (ITW) a key element in their cancer care programs [6].

Research opportunity

Quebec’s more than eight million residents enjoy a system of universal access to healthcare. The Quebec Cancer Program (PQLC – Programme québécois de lutte contre le cancer), publicly launched in 1998, was intended particularly to remedy deficiencies in service organization that had negative impacts both on the response to patients’ and families’ needs and on the quality and efficiency of care [7]. Since then, strategic plans have given priority to service reorganization, with a view to improving accessibility, continuity, person-centred care, and quality of care. This transformation has been supported by measures to promote better health services coordination and responsiveness. These include the creation of interdisciplinary teams in oncology outpatient clinics, the deployment of pivot nurses (also called nurse navigators in some parts of Canada), and the adoption of a person-centred approach [8]. The principal investigator of the proposed study and several members of the research team have conducted extensive research on health services transformation [916].

The PQLC, as other international cancer programs, supports the implementation of a strategy that would combine the efforts of all partners to produce the best possible response to the needs of persons with cancer. The PQLC’s 2007 to 2012 policy priorities recommended focusing on certain areas of intervention. ITW was presented as a solution to address the complexity of care and services required to meet the needs of persons with cancer [17].

Numerous efforts have been made and significant resources invested to encourage as many local oncology teams as possible to respond to these expectations. However, as we describe in greater detail below, the conclusions from evaluations of oncology care and services have clearly shown that ITW models vary from one team to another [3, 8, 18], as do the expected impacts on the care experience [9].

Generally speaking, studies on ITW (also called teamwork, collaborative practice, interprofessional practice) have concluded it produces benefits in terms of quality of patient care [1924]. A systematic review of the literature showed that better interprofessional and interorganizational collaboration could be associated with positive outcomes, both clinically (e.g., reduced mortality, shorter hospital stays, fewer readmissions) and in terms of patients’ perceptions of their care experience (e.g., greater satisfaction, improved quality of life) [25]. However, in-depth critical analysis of the evidence provides a more nuanced assessment of the causal links between ITW and its effects on the care experience. The term ‘care experience’ refers to patients’ perceptions regarding key aspects of oncology care and services, such as access to services, the quality of the response to their needs, and the effectiveness of care when they interact with service providers [26].

There are several reasons justifying the need for a more thorough study of the conditions underlying the production of ITW outcomes as they relate to the care experience. First, the concept of interdisciplinary teamwork goes by several different names (e.g., multidisciplinarity, interdisciplinary teamwork, collaborative practice, interprofessional collaboration), which are used interchangeably but for which there are subtle differences of definition [27, 28]. Because the subject being examined is not clearly defined and varies from one study to another, the result is a non-homogeneous body of knowledge from which it would seem rash to draw general conclusions on ITW outcomes for patients. Second, most of the instruments used in these studies to measure ITW focus on collaboration between physicians and nurses, whereas the outcomes may be different for teams with a more diverse professional composition [29], as is usually the case in oncology teams in Quebec. Third, some conclusions regarding outcomes related to ITW are also drawn from studies having to do with cancer therapy review committees (‘tumour boards’) [30, 31], or else with interdisciplinary teams of specialist physicians (oncologists, pathologists, internists, surgeons) [32]. These teams focus on treatment options according to the model of consultation among physicians in different specialties rather than on comprehensive management of the continuum of care based on a whole-person care practice model [33]. Consequently, the outcomes measured in the participants in these studies have to do only with cancer treatments, rather than with the full experience of care including treatments, as envisioned by the PQLC. Finally, most of these studies provide very little information on the characteristics of the teams and their operational models, whereas ITW is known to be context-dependent [34]. One common conclusion emerges from the literature review, which is that, regardless of the type of interdisciplinary team considered, the mechanisms by which ITW produces specific outcomes in oncology, as they emerge in natural settings, are currently receiving very little research attention and are poorly understood [25].

The proposed study will build upon a quasi-experimental study recently conducted in nine Quebec hospitals with 1,379 patients (referred to as Study 1 in this proposal). The results of that study revealed significant differences in certain ITW outcomes for patients according to the level of interdisciplinarity (high vs. low) in local teams [9]. These differences had to do with the time elapsed before accessing a pivot nurse, the ability to contact the team when a need arose, the possibility of being seen when there is a deterioration in health status, the response to overall needs, assistance and involvement with care decisions, and feelings of anxiety and distress. Using multivariate regression analysis, Study 1 also revealed that individual factors (e.g. male gender, age 70 years and over, lower education level, positive perception of one’s health status) and organizational factors (e.g. receiving care from a team consisting of eight different professionals or fewer, receiving care in a rural-area clinic) were associated with a more positive patient care experience. These results confirm that certain teams in Quebec manage to produce some of the desired outcomes of ITW, while others are less successful. However, given its quasi-experimental design [10], Study 1 was not able to capture the factors and sequencing of processes that would explain the variations in outcomes. Given the current lack of knowledge, the results of Study 1, and the efforts being made to promote and support ITW in local oncology teams in Quebec, it is becoming imperative to better understand the mechanisms by which outcomes are produced, by whom, for what groups of patients, and in what contexts.

The aim of the present study is to map out the conditions for the production of ITW outcomes in local oncology clinics in Quebec. More specifically, the objectives are: a) to identify the most critical contextual factors and mechanisms associated with production of ITW patient outcomes; b) to determine the synergetic or antagonistic influences of contextual factors and mechanisms on the production of ITW patient outcomes; and c) to explore the occurrence of unanticipated ITW mechanisms and patient outcomes.

ITW in oncology teams: intervention theory

Intervention theory refers to how the actors involved conceive of ITW in oncology teams, its functioning and its outcomes, as proposed by the PQLC. As part of the PQLC’s implementation, all local oncology teams in Quebec are expected to become involved in optimizing their ITW [11, 12, 17]. The aim is to ensure equity of access, without undue delays, to good-quality and safe care all along the cancer care continuum, from the first suspicion of illness to the period of survival or family bereavement, encompassing the entire period of active treatment and palliative care. ITW is a means of responding to complex and changing needs (of patients, families, and populations) by providing customized care centred on the whole person, at the right time, by the most appropriate professional, who is part of an oncology team. ITW relies on ‘synergy among practitioners (1 + 1 = 3)’ in a context of scarce resources, where the contributions of all professionals need to be optimized [13, 17]. Additional file 1 contains more details for describing the intervention (ITW in oncology teams) in order to improve the completeness of reporting and ultimately the transferability of the study results.

ITW in clinical teams: patient outcomes

In addition to the outcomes targeted by the PQLC, there are a number of other potential ITW outcomes identified in the literature. Despite the previously mentioned conceptual nuances characterizing the various terms applied to ITW, for the purposes of the following literature review we have adopted a pragmatic approach in which we consider them to be similar.

Overall, several studies have shown that ITW is associated with benefits for persons with chronic illnesses [1416, 35]. The results cover a wide range of observed outcomes, such as fewer clinical errors, complications, and hospitalizations, and shorter lengths of stay [3639]. Various studies also report positive impacts of ITW on patients’ and families’ satisfaction with services [40, 41], maintenance of functional capacity [40, 4244], compliance with treatment [44], greater accessibility to care, improved capacity for self-care, and healthy living habits, [43], health-promoting behaviours [43], and perceptions of better quality of care [45]. According to Schmitt and colleagues [43], ITW produces positive outcomes because it fosters a whole-person approach to the management of care. Other authors have associated positive outcomes with improved mechanisms for communication among professionals, who are increasingly working together toward a common goal [46]. However, there is no evidence that these benefits observed in other clienteles are transferable to patients with cancer. We also did not find any negative patient outcomes associated with ITW.

More specifically in oncology, a study in the United Kingdom explored the care experience of adolescents with cancer who were treated by an interdisciplinary team [47]. One conclusion of that qualitative study was that ITW in dedicated and specialized teams is a key element in creating a care environment that is responsive to the complex needs of this clientele. More recently, a study in a hospital in Spain showed that a higher level of ITW in the oncology care units was significantly associated with better pain management, higher levels of satisfaction among patients, and reduced uncertainty related to their illness [35]. That same study concluded that the intensity of collaboration had no impact on length of hospital stay. Our own work in Study 1 showed that ITW outcomes varied from one team to another. In some teams, the results confirmed our initial hypothesis of positive covariance between ITW level and the scope of effects for patients, i.e., that higher ITW levels would be accompanied by more positive patient perceptions of its effects on their experience. However, we found that, for certain aspects of the care experience, teams with lower ITW levels obtained results that were just as positive as those with higher ITW levels. Our results also showed that teams with a high level of ITW achieved outcomes that were less substantial than expected. While it is not clear why or how, it appears there may be one or more intermediate factors acting as moderator or mediator in the relationship between ITW and patient outcomes.

We should also point out that our results align with the conclusions of other authors regarding the need for further studies to better understand the links between ITW and its effects on quality of care. A recent review of the literature on ITW in oncology [33] reached the same conclusion as Lemieux-Charles and colleagues in 2006 [23], which was that studies establishing links between quality of team functioning and clinical effectiveness are few and far between, and offer inconsistent results. Some authors have criticized scientific journals’ tendency to publish only studies showing positive results of ITW [34], which constitutes a publication bias; they note the real absence of studies that observed either no effects or negative effects of ITW. According to these authors, this bias has meant that ITW is becoming a standard that is less and less subject to critical analysis. They also point out that many literature reviews on ITW in care teams have concluded that outcomes are positive without having evaluated the quality of the studies. Assessing the quality of the studies might reveal that the outcomes are less robust than is generally thought. There is also evidence that intra- and interprofessional collaborative practice is a best practice associated with credible patient outcomes [24]. Thus, given its nature and its effects, ITW can be approached as a complex intervention. According to the Medical Research Council, a complex intervention is characterized by several interdependent components, involves the practices of several actors in the healthcare system, and produces a variety of effects whose nature and scope can vary depending on the context [48]. Several authors agree that exploring these interventions more deeply presents very particular conceptual and methodological challenges [4951].

ITW in oncology teams: the challenges of evaluating complex interventions

The ‘realistic evaluation’ approach developed by Pawson and Tilley [52] is recognized as being useful for studying complex interventions when the aim is to go beyond determining whether an intervention is effective or not, and instead to explain how and why it is effective, under what conditions, and for which groups of patients. To be a ‘realistic evaluation’, the approach must be able to capture the contextualized action mechanisms at work in the relationship between an intervention and its effects, and to identify how these mechanisms are activated (or not). A mechanism refers to all ‘underlying entities, processes, or structures which operate in particular contexts to generate outcomes of interest’ [53]. Realistic evaluation is based on the theory: intervention context (C) + mechanism (M) = outcome (O), analyzed using a configurational approach. CMO configurations serve as a structure to identify what works (or not), how, by whom, with whom, for whom, and in what context. This theory is appropriate to guide our study because it links the conditions of production (C + M) and the outcomes (O) of an intervention according to a structured evaluation approach to what is actually happening in natural environments. It offers the possibility of deepening our understanding of ITW outcomes in oncology by identifying the synergetic or antagonistic effects in these associations among various contextual factors and action mechanisms that either foster or hinder the production of outcomes (Objectives 1 and 2 of the present study). The realistic evaluation approach is a pragmatic alternative to the experimental paradigm, given the impossibility of controlling complex interventions in natural environments, such as ITW [54]. Realistic evaluation thus offers a systematic investigative structure with two key advantages: it can take into account a multiplicity of variables and their relationships implied in the production of an intervention’s outcomes; and it can be used, at the same time, to minimize this complexity by modelling the relationships based on configurations of the three main dimensions: context–mechanisms–outcomes; both of these advantages facilitate the investigation and support potential action by users of the study’s results. Thus, contextual factors are considered not as variables to be controlled, as would be the case in experimental studies, but rather as intrinsic components of the phenomenon under study.

In addition to these theoretical foundations, realistic evaluation provides five key principles to consider in the research process: a) stakeholder participation in a study is required to understand – realistically – CMO configurations; b) mechanisms are theories developed based on propositions that ‘if things work this way in a given context, these will be the results’; c) CMO configurations may be established at the start, but they will be refined throughout the course of the study; d) causal links are generated as the study unfolds by determining how the variables are associated, rather than simply by measuring the correlation between a variable and a given outcome (as was done in Study 1); and e) the expected outcomes of an intervention are never guaranteed because of the central role played by the context [54, 55]. These realistic evaluation principles suggest that the first thing to do is to target the mechanisms that are likely to be active in the phenomenon before starting the study. In the following sections, realistic evaluation concepts are used to identify the contextual factors and mechanisms that are potentially involved in the production of ITW outcomes.

ITW in oncology teams: contextual factors

Those researchers who have a critical approach to ITW agree on the importance of context in explaining its variations. Context refers to the immediate and more distant environments whose structural characteristics shape care delivery [56]. As such, it is important to analyze context dynamically. Several contextual factors can contribute to the effectiveness of teams’ functioning [23, 57, 58]. These factors fall into three groups defined according to decision-making levels in the health system. The first group is at the practice (micro) level and encompasses the characteristics of professionals (e.g., profession, training, experience, leadership), patients’ clinical and sociodemographic characteristics (e.g., age, type of cancer, type of treatment, general health status, comorbidities), and team characteristics (e.g., composition, size, nature of mandate, work climate) [5961]. The second group is at the organizational (meso) level and includes institutional support, access to resources, leadership, etc. [15]. The third group is at the systemic (macro) level and comprises policies, national programs, legislative frameworks governing professional practice, etc. [62]. Each of these factors can exert variable influence in different CMO configurations, influences that will be more clearly delineated by the results of this present study.

ITW in oncology teams: mechanisms associated with outcomes production

Based on the ITW attributes described in the intervention theory coming out of the PQLC documents, it is possible to identify certain mechanisms that are potentially associated with the production of ITW outcomes in oncology. The first attribute – the bringing together of several practitioners – presents the challenge of sharing in the group the expertise, knowledge, and experience of professionals from different disciplines, who each have their own specific skills [34]. For a group of heterogeneous practitioners to rally around a common goal and collaborate, an effective communication mechanism must be in place and active [25, 57, 6366]. The second attribute – working together – requires a mechanism for care coordination to ensure continuity of care [25, 66]. When coordination is active, team members function interdependently, with each person’s contribution being important to optimize the management of information, clinical processes, and resources [67]. The third attribute – a global, shared, and unified understanding of the person – introduces the mechanism of patient-centred care [68]. Thus, instead of fragmented services centred around issues of professionalization, patients are presented with a person-centred care model in which an interdisciplinary care plan can be developed with objectives shared by the different practitioners on the team [69]. The fourth attribute –a concerted intervention that involves the sharing of complementary tasks – can refer both to a collaboration mechanism within the team and to integration between organizations [28, 70]. Finally, a fifth attribute appears to be essential for optimizing outcomes – evidence-based practice supported by a scientific knowledge utilization mechanism. Given the multidimensional nature of each of these mechanisms, it is worthwhile to characterize them here more clearly and to identify their potential effects based on the literature (see Additional file 2 for a detailed description and references).

ITW in oncology teams: interpretive framework

The intervention (ITW) coming out of the PQLC documents, the literature on ITW (definition, facilitating factors, constraining factors, outcomes), and the dimensions of realistic evaluation brings us to develop an interpretive framework (Figure 1).
Figure 1

Interpretive framework for evaluating the conditions for the production of ITW outcomes in oncology.

1) Programme québécoise de lutte contre le cancer (Quebec Cancer Program).

The framework thus incorporates three broad components surrounding ITW: a) the context, including characteristics of the environment, characteristics of the oncology clientele that could have an influence on the organization of the continuum of care and services, and characteristics of the organization and the team; b) mechanisms put forward to achieve ITW outcomes (communication, coordination, collaboration, person-centred care, negotiations and mutual adjustments, scientific knowledge utilization, and others that may be unforeseen); and c) the expected outcomes of ITW based on the intervention theory set out in the PQLC and the policy priorities as promised by the Quebec’s Department of Cancer Services. Using the data from Study 1, we will analyze the contextual factors and the mechanisms (C + M) associated with certain outcomes of interdisciplinarity that are comprised of several dimensions. We will analyze: responsiveness [71], which can be broken down into four subdimensions (promptness of access to care in response to patients’ needs, and the quality of communication, of the care environment, and of the person-centred response); continuity of care [72]; care outcomes [73] (management of symptoms and of health), as well as two subscales of the questionnaire on the capacity for self-care [74] (well-being and navigation of the healthcare system).

In light of these points, the key theories to be explored initially are:
  1. 1.

    Initial theory: that there is a positive relationship between ITW and certain aspects of the care experience as perceived by patients.

     
  2. 2.

    Alternative theory A: that there is a synergy (1 + 1 = 3) between certain contextual factors and certain mechanisms of team functioning that have a mediating influence on the relationship between ITW in oncology and patients’ perception of certain aspects of their care experience.

     
  3. 3.

    Alternative theory B: that there is an antagonistic association (1 + 1 = 1) between certain contextual factors and certain mechanisms of team functioning that have a moderating influence on the relationship between ITW in oncology and patients’ perception of certain aspects of their care experience.

     

Methods

Design

The study will use a retrospective multiple case study (n = 9) design [75]. Case study is an appropriate approach for characterizing clinical and organizational processes in their contexts [7577]. This is, in fact, the study design most often used by researchers investigating CMO configurations in the healthcare field [7881]. Each case consists of an interdisciplinary team in oncology working in the ambulatory clinic of a hospital. We will perform intra-case analyses to identify the CMO configurations, and then a cross-cutting inter-case analysis [82] to compare the cases. The study is retrospective, in that we will proceed in reverse to reconstruct the C + M associations that explain the outcomes measured in Study 1 (responsiveness, continuity, care outcomes, capacity for self-care) [83].

Case selection

The nine sites that participated in Study 1 will be invited to participate in the present study. For a multiple case study, the aim is not to have a sample that is statistically representative of a population, but rather to have the most informative cases possible, in which the features observed are likely to exist in other cases of populations with similar characteristics [76, 82, 84]. The diversity of the cases studied will allow us to observe recursive or singular CMO configuration models and to draw conclusions that could potentially be transferable to the different teams across Quebec.

Participants

The data, which will be qualitative, will be collected primarily by means of focus groups [85]. For each site, two different participant groups will be formed: G1 will be made up of the professionals in the oncology care team and the managers directly involved in the team’s deployment of ITW; G2 will consist of patients who have received services from that team. The number of participants per group will be around 6 to 10, perhaps going as high as 15 in certain cases, depending on the size of the local oncology team [86]. The criterion for inclusion in G1 is membership on the team for at least the past three years, to be able to provide an evolutionary perspective of the context and mechanisms put forward to support ITW since the start of Study 1. The criteria for inclusion in G2 (patients) are: a) to be comfortable with participating in a group discussion, and b) to have been in active treatment during the past year, such that they have had enough ‘exposure’ to the interdisciplinary team to be able to draw some conclusions about their care experience. They will be purposefully selected, with the help of a member of the local team, to ensure a diversity of sociodemographic characteristics (sex, age, education) and of clinical characteristics (type of treatment, type of cancer, phase of illness) [87]. The focus group approach was a strategic choice. It allows for spontaneous but focused discussion to better understand the conditions for the production of ITW outcomes from the perspectives of the persons most directly concerned. Focus groups provoke a ‘good conversation’ on the phenomenon being studied and make the best use of the limited time available to the group for in-depth consideration of the subject [88]. Particular attention will be paid to five criteria for assessing the success of a focus group: a) the intended topics are covered, and in sufficient depth; b) the statements elicited provide rich, diverse and nuanced testimony; c) participation is active, spontaneous and balanced; d) there is interaction and mutual influence, but no excessive dominance by any participants; and e) the group process contributes to reflection that allows participants to embark in a change process [85]. In this way, we will be able to cover all of the potential explanatory factors for the production of ITW outcomes by starting from the outcomes and working backwards to investigate the conditions (context and mechanisms) that determine them [83] (Objectives 1 and 2), while the spontaneity and dynamic nature of the group will enrich the explanations and help to identify unanticipated mechanisms and outcomes (Objective 3).

The focus group data will be triangulated with complementary data gleaned from the relevant documentation (meeting minutes, medical records charting tools, inter-professional and inter-organizational referral tools) and from informal discussions [82].

Data collection

Data will be systematically collected in the focus groups with professionals and patients by means of pre-defined questions presented in the focus group meeting plan (see Additional file 3). The groups will be facilitated by the co-investigators, who have extensive experience in oncology and in leading oncology team meetings. The professionals will be invited to reflect on the conditions that might explain the ITW outcomes in their setting. These outcomes, measured in Study 1, are presented in the local reports distributed to the teams in 2012. Those reports summarize the context of Study 1 and its methodology, and present descriptive statistics from the responses to the questionnaire on accessibility, responsiveness, continuity, capacity for self-care, and characteristics of respondents. The report presenting both local and global results (aggregated for the nine sites) will be used as an intermediary object [89] in a reflexive process (learning about and in practice) [90] and as a means of encouraging discussion. For items that characterize the greatest variation in the care experience of patients receiving care from the local team, the professionals will take a position on what they think about the results, as well as on the gaps between local and global results and how they might be explained: what mechanisms facilitate the production of ITW outcomes; what mechanisms contribute to maintaining practices that are less interdisciplinary; and by whom, with whom, and for whom outcomes are produced, and in what specific contexts.

With regard to the patient focus groups, the local report is considered too abstract to provoke any discussion on the care experience. As such, the intermediary object chosen for that purpose will be a vignette covering the active treatment period. Used in research, a vignette is a brief scenario presented either in writing or through pictograms, which serves as a proxy for a real situation, and to which participants are invited to respond [91]. The vignette has several advantages: it helps depersonalize the discussion while keeping the focus on the phenomenon under study, and it is a standardized, flexible, and inexpensive tool. As we did in an earlier study to assess the quality of care in oncology [92], we will reconstruct the cancer care continuum of a patient with colon cancer (post-surgery diagnosis, first phase of chemotherapy, progression of the illness, and decision to pursue more aggressive treatment), inserting certain items that showed the greatest variation in ITW outcomes when measured in Study 1. For example, the vignette will address items related to responsiveness by illustrating such things as participation in decisions about care, help in weighing the pros and cons of treatment, family involvement, and support in managing anxiety when dealing with illness.

The questions used to structure the discussion are inspired by realistic evaluation and are aimed not only at identifying ITW outcomes (the ‘what’), but also at explaining how and why those outcomes are produced. At each focus group meeting, we will explain the process we intend to follow, then present a summary of the results of Study 1, followed by a discussion as presented in the focus group outline provided in the Additional file 3[93]. The discussions will be digitally recorded and then transcribed in their entirety for analysis. All the material will be entered into a database using QDA Miner software [94].

Data analysis

The data will be analyzed using an iterative content analysis process with systematic coding [95, 96]. The analysis will be structured based on the 10 steps developed by Ely [97]. First, we will develop a semi-structured coding grid based on the concepts of our interpretive framework (context, mechanisms, outcomes) leaving open the possibility of adding emerging concepts to the CMO configurations over the course of the analysis [98]. The data for each case will be analyzed separately. Then we will perform an inter-case analysis to identify similar and different CMO configurations in the different cases. According to Campbell [99], certain case characteristics may only become evident through inter-case comparisons. Finally, we will use a process of iteration between the intra-case and inter-cases analyses to ensure that we are able to satisfy the objectives of the study and to produce enough evidence to facilitate the transferability of our results to other organizations whose characteristics are similar to those in our study cases. To ensure the quality of the analysis, the members of the research team will discuss the coding process and results among themselves and also with collaborators and key stakeholders (members of professional teams, managers, decision-makers) in validation meetings. If we are able to identify patients who would be at ease and willing to participate in such validation discussions, they will be included.

The study has been approved by the Research Ethics Board of the Charles-LeMoyne Hospital Research Centre (ref. number MP-HCLM-13-034).

Study validity

In this study, we will pay special attention to ensuring both internal and external validity. Internal validity refers to the degree of similarity and plausibility that exists between the knowledge construction process, the complexity of a phenomenon in its natural setting, and the results of a study [95]. We have employed several means to ensure the internal validity of our study: using cases whose characteristics we know well from having worked with them in Study 1 [75, 83]; incorporating an extensive review of the ITW literature; building on the theoretical foundations and methodologies of realistic evaluation [100]; and triangulating several data sources [101], i.e., the perspectives of professionals and patients, as well as documentary analysis. We will triangulate data sources using two strategies that highlight different aspects of the conditions for production of ITW outcomes: a) the comparison of similarities and differences in the perspectives of the various actors with regard to the context (C) and mechanisms (M) of interdisciplinary care and the C + M links in the proposed study; and b) the validation of data from the interviews as compared with the data from documentary sources [87]. In our proposed study, the solidity of the links established between context, mechanisms and outcomes is reinforced by the study of multiple cases [82] and by the complementarity of the quantitative and qualitative research approaches of Study 1 and the present study [102, 103]. Moreover, Rohlfing (2008) asserts that, by taking outcomes as the starting point and working backward to investigate the conditions that determine them, we are more likely to capture the whole range of causes that might have influenced the outcomes [83]. With regard to external validity, it has to do with the degree to which a study’s results can be generalized to other populations, settings, and time periods having similar characteristics [104]. However, case study, in the context of realistic evaluation, does not aim for quantitative or statistical generalizability, but rather to provide plausible explanations obtained from a sample in which, ‘if things work this way, these will be the results.’ These explanations can be generalized to other contexts where the organizational and environmental conditions are similar [95].

Discussion

To the best of our knowledge, this study is the first to use CMO associations to improved empirical and theoretical understanding of interdisciplinary teamwork in oncology, and its results could foster more effective implementation in clinical practice. This study will have significant impacts because the results will foster innovation by helping to fill the gaps in knowledge about the strongly contextualized mechanisms responsible for differences in ITW outcomes in oncology. It will be possible to identify which are the most critical mechanisms to activate, to determine whether certain mechanisms are more important than others, and to specify in which contexts these mechanisms are most active. With a better understanding of these factors, it will be possible to target interventions to act on the determining conditions in ways that optimize positive outcomes. Settings with less favourable conditions can be better supported in their efforts to optimize ITW. Given the study design, the results will be useful for different types of actors at different levels of the healthcare system, i.e., clinicians, managers, and decision-makers who determine strategic actions in the PQLC’s policy priorities. These new data will be especially important in a context where every professional’s contribution will need to be optimized, to contend with the rising incidence of cancer as the population ages and the limited resources of the healthcare system.

Abbreviations

CMO: 

Context–mechanisms–outcomes

ITW: 

Interdisciplinary teamwork

PQLC: 

Programme québécois de lutte contre le cancer/Quebec cancer program.

Declarations

Acknowledgements

This study is funded by the Fonds de recherche du Québec - Santé (FRQ-S – Grant # 27193) for which DT is the principal investigator. We thank Danielle Drouin for comments in the vignette development and Michèle Rietmann for assistance provided in the preparation of this manuscript.

Authors’ Affiliations

(1)
Charles-Le Moyne Hospital Research Centre
(2)
Faculty of Medicine and Health Sciences, Université de Sherbrooke-Campus Longueuil
(3)
École Nationale d’Administration Publique
(4)
CSSS Champlain–Charles-Le Moyne

References

  1. Howell D, Currie S, Mayo S, Jones G, Boyle M, Hack T, Green E, Hoffman L, Simpson J, Collacutt V, McLeod D, Digout C: A Pan-Canadian Clinical Practice Guideline: Assessment of Psychosocial Health Care Needs of the Adult Cancer Patient. 2009, Toronto: Canadian Partnership Against Cancer (Cancer Journey Action Group) and the Canadian Association of Psychosocial Oncology,http://www.capo.ca/Fatigue_Guideline.pdf,Google Scholar
  2. Comité des représentants de la population atteinte de cancer et des proches (REPOP): Besoins des personnes atteintes de cancer et de leurs proches au Québec, Recommandations, Avis. 2005, [http://www.merici.ca/Bibliotheque_pdf/msss_quebec/besoins_pers_cancer.pdf]Google Scholar
  3. La lutte contre le cancer dans les régions du Québec - Un premier bilan.http://msssa4.msss.gouv.qc.ca/fr/document/publication.nsf/fb143c75e0c27b69852566aa0064b01c/a32c2ea1f943a28485256e7b00504064?OpenDocument,
  4. Morrison V, Henderson BJ, Zinovieff F, Davies G, Cartmell R, Hall A, Gollins S: Common, important, and unmet needs of cancer outpatients. Eur J Oncol Nurs. 2012, 16: 115-123.PubMedGoogle Scholar
  5. Thorne SE, Bultz BD, Baile WF: Is there a cost to poor communication in cancer care?: a critical review of the literature. Psycho-Oncology. 2005, 14: 875-884.PubMedGoogle Scholar
  6. Haward R, Amir Z, Borrill C, Dawson J, Scully J, West M, Sainsbury R: Breast cancer teams: the impact of constitution, new cancer workload, and methods of operation on their effectiveness. Brit J Cancer. 2003, 89: 15-22.PubMedPubMed CentralGoogle Scholar
  7. Comité consultatif sur le cancer: Programme québécois de lutte contre le cancer. Pour lutter efficacement contre le cancer, formons équipe. 1997,http://publications.msss.gouv.qc.ca/acrobat/f/documentation/1997/97-729-5.pdf,Google Scholar
  8. Le bilan des réalisations des orientations prioritaires 2007–2012 du Programmequébécois de lutte contre le cancer.http://publications.msss.gouv.qc.ca/acrobat/f/documentation/2011/11-902-16.pdf,
  9. Tremblay D, Roberge D, Cazale L, Touati N, Maunsell E, Latreille J, Lemaire J: Evaluation of the impact of interdisciplinarity in cancer care. BMC Health Serv Res. 2011, 11: 144PubMedPubMed CentralGoogle Scholar
  10. Champagne F, Brousselle A, Hartz Z, Contandriopoulos A-P, Denis J-L: L’analyse de l’implantation. L’évaluation: concepts et méthodes. Edited by: Brousselle A, Champagne F, Contandriopoulos A-P, Hartz Z. 2011, Montréal: Les Presses de l’Université de Montréal, 237-273.Google Scholar
  11. Comité des équipes interdisciplinaires de lutte contre le cancer (CEILCC): Les équipes interdisciplinaires en oncologie. 2005, Avis, [http://www.msss.gouv.qc.ca/sujets/prob_sante/cancer/download.php?f=e899a549fb5483bca704b9f28a7e03ec]Google Scholar
  12. Direction de la lutte contre le cancer, Ministère de la Santé et des Services sociaux du Québec, Gouvernement du Québec: Rapport d’activité 2008–2009. 2009,http://publications.msss.gouv.qc.ca/acrobat/f/documentation/2009/09-902-02.pdf,Google Scholar
  13. Partenariat canadien contre le cancer: Stimuler l’innovation dans la prestation des services: Symposium sur l’optimisation des professionnels de la santé contre le cancer. 2010,http://www.partnershipagainstcancer.ca/sites/default/files/CPAC%20Symposium%20FRE%20Report%20Finalr3.pdf,Google Scholar
  14. Simmonds S, Cold J, Joseph P, Marriott S, Tyrer P: Community mental health team management in severe mental illness: a systematic review. Br J Psychiat. 2001, 178: 497-502.Google Scholar
  15. Eccles MP, Hawthorne G, Johntson M, Hunter M, Steen N, Francis J, Hrisos S, Elovainio M, Grimshaw JM: Improving the delivery of care for patients with diabetes through understanding optimised team work and organisation in primary care. Implement Sci. 2009, 4: 22PubMedPubMed CentralGoogle Scholar
  16. Schofield RF, Amodeo M: Interdisciplinary teams in health care and human services settings: are they effective?. Health Soc Work. 1999, 24: 210-219.PubMedGoogle Scholar
  17. Direction de la lutte contre le cancer, Ministère de la Santé et des Services sociaux du Québec, Gouvernement du Québec: Orientations prioritaires 2007–2012 du Programme québécois de lutte contre le cancer. 2007, [http://publications.msss.gouv.qc.ca/acrobat/f/documentation/2007/07-902-03.pdf]Google Scholar
  18. Agence d’évaluation des technologies et des modes d’intervention en santé(AETMIS): L’évaluation externe de l’organisation et de la prestation des soins en oncologie. Revue des expériences pertinentes pour le processus d’évaluation et de désignation des établissements et des équipes interdisciplinaires de lutte contre le cancer au Québec, 2010. ETMIS. 2010, 6 (6): 1-152. [http://www.inesss.qc.ca/fileadmin/doc/AETMIS/Rapports/Cancer/ETMIS2010_Vol6_No6.pdf]Google Scholar
  19. Health Professions Regulatory Advisory Council (HPRAC): Interprofessional Collaboration: A Summary of Key Reference Documents & Selected Highlights from the Literature. 2008, Toronto,http://www.hprac.org/en/projects/resources/hprac-collaboration.LitReviewENFINAL.feb1208.pdf,Google Scholar
  20. Thornhill J, Dault M, Clements D: Ready, set … collaborate? the evidence says “Go,” so what’s slowing adoption of inter-professional collaboration in primary healthcare?. Healthc Q. 2008, 11: 14-16.Google Scholar
  21. Canadian health services research foundation: Teamwork in healthcare: promoting effective teamwork in healthcare in Canada. Policy synthesis and recommendations. 2006, Ottawa,http://www.cfhi-fcass.ca/Migrated/PDF/teamwork-synthesis-report_e.pdf,Google Scholar
  22. World Health Organization: Framework for Action on Interprofessional Education & Collaborative Practice. 2010,http://whqlibdoc.who.int/hq/2010/WHO_HRH_HPN_10.3_eng.pdf,Google Scholar
  23. Lemieux-Charles L, McGuire W: What do we know about health care team effectiveness? a review of the literature. Med Care Res Rev. 2006, 63: 263-300.PubMedGoogle Scholar
  24. Registered Nurses’ Association of Ontario (RNAO): Healthy Work Environments Best Practice Guidelines, Collaborative Practice AmongNursing Teams. 2006, [http://rnao.ca/sites/rnao-ca/files/Collaborative_Practice_Among_Nursing_Teams.pdf]Google Scholar
  25. Gagliardi AR, Dobrow MJ, Wright FC: How can we improve cancer care? A review of interprofessional collaboration models and their use in clinical management. Surg Oncol. 2011, 20: 146-154.PubMedGoogle Scholar
  26. Canadian Partnership Against Cancer: Sustaining Action Toward a Shared Vision. 2012, [http://www.partnershipagainstcancer.ca/wp-content/uploads/Sustaining-Action-Toward-A-Shared-Vision_web.pdf]Google Scholar
  27. Choi BC, Pak AW: Multidisciplinarity, interdisciplinarity and transdisciplinarity in health research, services, education and policy: 1. definitions, objectives, and evidence of effectiveness. Clin Invest Med. 2006, 29: 351-364.PubMedGoogle Scholar
  28. D’Amour D, Ferrada-Videla M, San Martin-Rodriguez L, Beaulieu MD: The conceptual basis for interprofessional collaboration: core concepts and theoretical frameworks. J Interprof Care. 2005, 19: 116-131.PubMedGoogle Scholar
  29. Kenaszchuk C, Reeves S, Nicholas D, Zwarenstein M: Validity and reliability of a multiple-group measurement scale for interprofessional collaboration. BMC Health Serv Res. 2010, 10: 1-15.Google Scholar
  30. Petty JK, Vetto JT: Beyond doughnuts: tumor board recommendations influence patient care. J Cancer Educ. 2002, 17: 97-100.PubMedGoogle Scholar
  31. Gray JR: The tumor conference: an integral component of the oncology program. J Oncol Manag. 1997, 6: 10-14.PubMedGoogle Scholar
  32. Castel P, Friedberg E: Institutional change as an interactive process: the modernization of the French cancer centers. Organ Sci. 2010, 21: 311-330.Google Scholar
  33. Fennell ML, Prabhu Das I, Clauser S, Petrelli N, Salner A: The organization of multidisciplinary care teams: modeling internal and external influences on cancer care quality. J Natl Cancer Inst Monogr. 2010, 2010: 72-80.PubMedPubMed CentralGoogle Scholar
  34. Reeves S, Lewin S, Espin S, Zwarenstein M: Interprofessional teamwork in health and social care. 2010, Oxford: Wiley-BlackwellGoogle Scholar
  35. San Martin Rodriguez L: Ph.D. thesis. Évaluation des effets de la collaboration interprofessionnelle chez les professionnels et chez les patients dans les unités d’hospitalisation en oncologie et en hématologie. 2007, Canada: Université de MontréalGoogle Scholar
  36. McAlister FA, Stewart S, Ferrua S, McMurray JJ: Multidisciplinary strategies for the management of heart failure patients at high risk for admission: a systematic review of randomized trials. J Am Coll Cardiol. 2004, 44: 810-819.PubMedGoogle Scholar
  37. Naylor CJ, Griffiths RD, Fernandez RS: Does a multidisciplinary total parenteral nutrition team improve outcomes? a systematic review. JPEN J Parenter Enteral Nutr. 2004, 28: 251-258.PubMedGoogle Scholar
  38. Whetten DA, Cameron KS: Building effective teams and teamwork. Developing management skills. Edited by: Whetten DA, Cameron KS. 2007, Jersey City, NJ: Prentice Hall, 493-535.Google Scholar
  39. Silas L: From promise to practice: getting healthy work environments in health workplaces. Healthcare Papers. 2007, 7: 46-51.PubMedGoogle Scholar
  40. Schmitt MH: Collaboration improves the quality of care: methodological challenges and evidence from US health care research. J Interprof Care. 2001, 15: 47-66.PubMedGoogle Scholar
  41. Zwarenstein M, Bryant W: Interventions to promote collaboration between nurses and doctors. Cochrane DB Syst Rev. 2000, 2: CD000072Google Scholar
  42. Freeth D: Sustaining interprofessional collaboration. J Interprof Care. 2001, 15: 37-46.PubMedGoogle Scholar
  43. Schmitt MH, Farrell MP, Heinemann GD: Conceptual and methodological problems in studying the effects of interdisciplinary geriatric teams. Gerontologist. 1988, 28: 753-764.PubMedGoogle Scholar
  44. Halstead LS: Team care in chronic illness: a critical review of the literature of the past 25 years. Arch Phys Med Rehabil. 1976, 57: 507-511.PubMedGoogle Scholar
  45. Finn R, Currie G, Martin G: Team work in context: institutional mediation in the public-service professional bureaucracy. Organ Stud. 2010, 31: 1069-1097.Google Scholar
  46. Baldwin LM, Taplin SH, Friedman H, Moe R: Access to multidisciplinary care: is it linked to the use of breast-conserving surgery with radiation for early-stage breast carcinoma?. Cancer. 2004, 100: 701-709.PubMedGoogle Scholar
  47. Mulhall A, Kelly D, Pearce S: A qualitative evaluation of an adolescent cancer unit. Eur J Cancer Care. 2004, 13: 16-22.Google Scholar
  48. Medical Research Council: Developing and Evaluating Complex Interventions: New Guidance. 2008, London, [http://www.mrc.ac.uk/documents/pdf/complex-interventions-guidance/]Google Scholar
  49. Campbell M, Fitzpatrick R, Haines A, Kinmonth A, Sandercock P, Spiegelhalter D, Tyrer P: Framework for design and evaluation of complex interventions to improve health. Brit Med J. 2000, 321: 694-696.PubMedPubMed CentralGoogle Scholar
  50. Campbell NC, Murray E, Darbyshire J, Emery J, Farmer A, Griffiths F, Guthrie B, Lester H, Wilson P, Kinmonth AL: Designing and evaluating complex interventions to improve health care. Brit Med J. 2007, 334: 455-459.PubMedPubMed CentralGoogle Scholar
  51. Murray E, Treweek S, Pope C, MacFarlane A, Ballini L, Dowrick C, Finch T, Kennedy A, Mair F, O’Donnell C, Ong BN, Rapley T, Rogers A, May C: Normalisation process theory: a framework for developing, evaluating and implementing complex interventions. BMC Med. 2010, 8: 63PubMedPubMed CentralGoogle Scholar
  52. Pawson R, Tilley N: Realistic evaluation. 1997, London: Sage PublicationsGoogle Scholar
  53. Astbury B, Leeuw FL: Unpacking black boxes: mecanisms and theory building in evaluation. Am J Eval. 2010, 31: 363-381.Google Scholar
  54. Pawson R, Tilley N: Realist Evaluation. 2004,http://www.communitymatters.com.au/RE_chapter.pdf,Google Scholar
  55. Kazi M: Realist evaluation in practice: health and social work. 2003, Thousand Oaks: Sage PublicationsGoogle Scholar
  56. McCormack B, Kitson A, Harvey G, Rycroft-Malone J, Titchen A, Seers K: Getting evidence into practice: the meaning of context. J Adv Nurs. 2002, 38: 94-104.PubMedGoogle Scholar
  57. Borrill C, Carletta JC, Carter A, Dawson JF, Garrod S, Rees A, Richards A, Shapiro D, West M: Team Working and Effectiveness in Health Care: Findings from the Health Care Team Effectiveness Project. 2001, [http://homepages.inf.ed.ac.uk/jeanc/DOH-glossy-brochure.pdf]Google Scholar
  58. Heinemann GD, Zeiss AM: Team performance in health care: assessment and development. 2002, New York: SpringerGoogle Scholar
  59. Willard C, Luker K: Working with the team: strategies employed by hospital cancer nurse specialists to implement their role. J Clin Nurs. 2007, 16: 716-724.PubMedGoogle Scholar
  60. Maslin-Prothero SE, Bennion EA: Integrated team working: a literature review. Int J Integr Care. 2010, 10: e043PubMedPubMed CentralGoogle Scholar
  61. Xyrichis A, Lowton K: What fosters or prevents interprofessional teamworking in primary and community care? a literature review. Int J Nurs Stud. 2008, 45: 140-153.PubMedGoogle Scholar
  62. Lemieux-Charles L: Understanding the conditions that lead to effective health services delivery networks. Healthc Pap. 2006, 7: 40-45.PubMedGoogle Scholar
  63. Akrich M, Callon M, Latour B: A quoi tient le succès des innovations. 2: L’art de choisir les bons porte-parole. Gérer et Comprendre, Annales des Mines. 1988, 12: 14-29.Google Scholar
  64. San Martin Rodriguez L, Beaulieu M, D’Amour D, Ferrada M: The determinants of successful collaboration: a review of theoretical and empirical studies. J Interprof Care. 2005, 19: 132-147.PubMedGoogle Scholar
  65. Begun JW, White KR, Mosser G: Interprofessional care teams: the role of the healthcare administrator. J Interprof Care. 2011, 25: 119-123.PubMedGoogle Scholar
  66. Stewart GL: A meta-analytic review of relationships between team design features and team performance. J Manage. 2006, 32: 29-55.Google Scholar
  67. Campion MA, Medsker GJ, Higgs AC: Relations between work group characteristics and effectiveness: implications for designing effective work groups. Pers Psychol. 1993, 46: 823-850.Google Scholar
  68. Mead N, Bower P: Patient-centredness: a conceptual framework and review of the empirical literature. Soc Sci Med. 2000, 51: 1087-1110.PubMedGoogle Scholar
  69. D’Amour D, Tremblay D, Bernier L: Les pratiques professionnelles de réseaux: l’intégration au delà des structures. Le système sociosanitaire au Québec: gouverne, régulation et participation. 2006, Boucherville: Gaétan Morin, 273-287.Google Scholar
  70. Powell AE, Davies HTO: The struggle to improve patient care in the face of professional boundaries. Soc Sci Med. 2012, 75: 807-814.PubMedGoogle Scholar
  71. Tremblay D: Ph.D. thesis. La traduction d’une innovation organisationnelle dans les pratiques professionnelles de reseau: l’infirmiere pivot en oncologie. 2008, Canada: Universite de MontrealGoogle Scholar
  72. Levesque J-F, Pineault R, Simard B, Hamel M, Roberge D, Ouellet D, Haggerty J: Questionnaire populationnel, L’accessibilité et la continuité des services de santé : une étude sur la première ligne au Québec. 2006, [http://www.greas.ca/evolution/pdf/questionnairepopfr.pdf]Google Scholar
  73. Conroy T, Mercier M, Bonneterre J, Luporsi E, Lefebvre JL, Lapeyre M, Puyraveau M, Schraub S: French version of FACT-G: validation and comparison with other cancer-specific instruments. Eur J Cancer. 2004, 40: 2243-2252.PubMedGoogle Scholar
  74. Osborne RH, Elsworth GR, Whitfield K: The health education impact questionnaire (heiQ): an outcomes and evaluation measure for patient education and self-management interventions for people with chronic conditions. Patient Educ Couns. 2007, 66: 192-201.PubMedGoogle Scholar
  75. Gerring J: Case study research: principles and practices. 2007, New York: Cambridge University PressGoogle Scholar
  76. Fitzgerald L, Dopson S: Comparative case study designs: their utility and development in organizational research. The SAGE handbook of organizational research methods. Edited by: Buchanan DA, Bryman A. 2009, Thousand Oaks: SAGE Publications, 465-483.Google Scholar
  77. Gerring J: What is a case study and what is it good for?. Am Polit Sci Rev. 2004, 98: 341-354.Google Scholar
  78. Greenhalgh T, Humphrey C, Hughes J, Macfarlane F, Butler C, Pawson R: How do you modernize a health service? a realist evaluation of whole-scale transformation in London. Milbank Q. 2009, 87: 391-416.PubMedPubMed CentralGoogle Scholar
  79. Rycroft-Malone J, Fontenla M, Bick D, Seers K: A realistic evaluation: the case of protocol-based care. Implement Sci. 2010, 5: 38PubMedPubMed CentralGoogle Scholar
  80. Redfern S, Christian S, Norman I: Evaluating change in health care practice: lessons from three studies. J Eval Clin Pract. 2003, 9: 239-249.PubMedGoogle Scholar
  81. Kennedy A, Rogers A, Gately C: Assessing the introduction of the expert patients programme into the NHS: a realistic evaluation of recruitment to a national lay-led self-care initiative. Prim Health Care Res Dev. 2005, 6: 137-148.Google Scholar
  82. Yin RK: Case study research: design and methods. 2009, Thousand Oaks: Sage Publications, 4Google Scholar
  83. Rohlfing I: What you see and what you get: pitfalls and principles of nested analysis in comparative research. Comp Polit Stud. 2008, 41: 1492-1514.Google Scholar
  84. Eisenhardt KM: Building theories from case study research. Acad Manage Rev. 1989, 14: 532-550.Google Scholar
  85. Leclerc C, Bourassa B, Picard F, Courcy F: Du groupe focalisé à la recherche collaborative: avantages, défis et stratégies. Recherche qualitative. 2010, 29: 145-167.Google Scholar
  86. Morgan DL: Focus groups as qualitative research. 1997, Thousand Oaks: Sage Publications, 2Google Scholar
  87. Patton MQ: Qualitative research & evaluation methods. 2003, Thousand Oaks: Sage Publications, 3Google Scholar
  88. Kitzinger J, Marková I, Kalampalikis N: Qu’est-ce que les focus groups?. B Psychol. 2004, 57: 237-243.Google Scholar
  89. Vinck D: Les objets intermédiaires dans les réseaux de coopération scientifique – contribution à la prise en compte des objets dans les dynamiques sociales. Rev Fr Sociol. 1999, 40: 385-414.Google Scholar
  90. Schön DA: The reflective practitioner: how professionals think in action. 1983, New York: Basic BooksGoogle Scholar
  91. Finch J: The vignette technique in survey research. Sociology. 1987, 21: 105-114.Google Scholar
  92. Cazale L, Tremblay D, Roberge D, Touati N, Denis J-L, Pineault R: Développement et application d’une vignette clinique pour apprécier la qualité des soins d’équipes interdisciplinaires en oncologie. Rev Epidemiol Sante. 2006, 54: 407-420.Google Scholar
  93. Van der Maren J-M: La maquette d’un entretien: son importance dans le bon déroulement de l’entretien et dans la collecte de données de qualité. Recherches qualitatives. 2010, 29: 129-139.Google Scholar
  94. Lewis RB, Maas SM: QDA miner 2.0: mixed-model qualitative data analysis software. Field Methods. 2007, 19: 87-108.Google Scholar
  95. Miles MB, Huberman AM, Saldana J: Qualitative data analysis: a methods sourcebook. 2014, Thousand Oaks: Sage Publications, 3Google Scholar
  96. Saldana J: The coding manual for qualitative researchers. 2009, Thousand Oaks: Sage PublicationsGoogle Scholar
  97. Ely M, Anzul M, Friedman T, Gardner D, McCormack-Steinmet A: Doing qualitative research: circles within circles. 1991, New York: RoutledgeGoogle Scholar
  98. Van der Maren JM: Méthodes de recherche pour l’éducation. 1996, Montréal: Presses de l’Université de Montréal, 2Google Scholar
  99. Campbell S: Comparative case study. Encyclopedia of case study research. Edited by: Mills AJ, Durepos G, Wiebe E. 2010, Thousand Oaks: Sage Publications, 174-175.Google Scholar
  100. Whittemore R, Chase SK, Mandle CL: Validity in qualitative research. Qual Health Res. 2001, 11: 522-537.PubMedGoogle Scholar
  101. Apostolidis T: Representations sociales et triangulation: une application en psychologie sociale de la sante. Psic: Teor e Pesq. 2006, 22: 211-226.Google Scholar
  102. Morgan DL: Paradigms lost and pragmatism regained: methodological implications of combining qualitative and quantitative methods. J Mixed Method Res. 2007, 1: 48-76.Google Scholar
  103. Wolf F: Enlightened eclecticism or hazardous hotchpotch? mixed methods and triangulation: strategies in comparative public policy research. J Mixed Method Res. 2010, 4: 144-167.Google Scholar
  104. Cook TD, Campbell DT: Quasi-experimentation: design and analysis issues for field settings. 1979, New York: Houghton Mifflin CompanyGoogle Scholar

Copyright

© Tremblay et al.; licensee BioMed Central Ltd. 2014

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Advertisement