This research and development project will be carried out in five phases.
Phase 1: identification and critical appraisal of existing instruments
Of the 78 studies included in a systematic review of studies of factors influencing health professionals' behaviour, we identified 49 used tools with acceptable psychometric qualities . These studies will be retrieved for details about the instruments in question. Several methods will be used to ensure the completeness of the information regarding each instrument. If the paper does not provide detailed information, the authors will be contacted. The expected outcome of this phase is an inventory of instruments based on social cognitive theories. This initial inventory will be validated and enriched by updating the literature search and contacting experts in the field (CPD specialists, health services researchers, and social psychologists) through diverse mailing lists.
The instruments thus identified will be analyzed using criteria based on the literature on measurement development (e.g., instructional design principles requirement, psychometric properties) , , . They will also be analyzed based on criteria that CPD decision makers deem important (e.g., be able to be used by a large number of physicians in different clinical areas and contexts and be inexpensive to administer). To fulfill these tasks, two research assistants will independently extract data using a standardized form that will be revised at the beginning of the research project by team members to ensure that it fits decision makers' needs.
Phase 2: assessment of the instrument's applicability--development of a new global instrument based on social cognitive theories
The goal of phase 2 is to assess the applicability of and adapt any instruments identified in phase 1. We will create a seven-person Development Committee comprised of researchers, CPD decision makers, and an external expert from each of the CPD and knowledge-translation research communities. Members of this committee will largely be from the research team. Examining each instrument selected in phase 1, the Development Committee will choose the most relevant instruments and their related items for assessing the variables (concepts) listed in our integrated conceptual model. Once this task is complete and all items are integrated into a new global instrument, an e-Delphi process will be used to check its face validity and likely acceptability and utility in CPD settings. Acceptability will be assessed with criteria used in phase 1. The resulting global instrument will also be examined for (a) its appropriateness or relevance to behaviours expected to occur after CPD activities, (b) its grammar and readability, and (c) the appearance of bias in two rounds of planned contacts within the entire research team and its network.
Once the team and its networks have provided their feedback, the Development Committee will review the final list of selected instruments and their respective items grouped around variables (concepts) of the integrated conceptual model and ensure that the new global instrument (i.e., one that includes selected instruments) continues to meet the criteria elaborated above. The Development Committee may revise or discard items and variables as necessary. We will begin by developing a French version of the global instrument.
Phase 3: assessment of the reliability and validity of the new theory-based global instrument
The objective of phase 3 is to assess the reliability and validity of the new theory-based global instrument, including its ability to predict intentions and behaviour, and also its sensitivity to change in response to CPD activities in a group of physicians who have participated in an accredited CPD activity.
Study population and recruitment strategy
Participants will be recruited from attendees of the CPD activities offered by the CPD decision makers and collaborators to this project. Each year, thousands of educational activities are offered through this network. These institutions and organisations reach the vast majority of the 16,000 practising physicians in the province of Québec. Eligibility criteria for CPD activities will include the following: (a) accreditation by one of the CPD decision makers and collaborators to this project; (b) group-based, live activities carried out with groups of 50 participants or less; (c) a focus on behaviour change for any topic or content as stated in the learning objectives established for the activity; (d) occurrence in any setting (e.g., university, conference venue, practice setting); (e) use of any or a combination of instructional methods (e.g., didactic lectures, workshops, case studies, demonstrations) and material (e.g., audience response systems, videos, card games, real or simulated patients), which must include at least 25% of interactivity; (f) conduct as a one-time intervention; and (g) duration of between at least one and three hours. Individual activities embedded within large programs offering several activities at one setting, such as a two-day conference, will be eligible.
Eligibility criteria for physicians will include the following: (a) attending an eligible live CPD activity, (b) being active in clinical practice for a six-month period following the indexed CPD activity, (c) being accessible for a phone interview three months after the indexed CPD activity, (d) speaking fluent French, and (e) not having participated in this study previously.
A research assistant will attend a number of eligible CPD activities chosen at random from one of the CPD decision-makers' and collaborators' networks. At registration in the CPD activity, the assistant will enroll participants and ask them to complete a consent form and a sociodemographic questionnaire. The research assistant will also collect data on the CPD activity to which the participants are exposed.
Participant-level data collected for the study will include the participants' sociodemographic details (e.g., gender, age, type of medical speciality, and years of experience in current practice). CPD activity-level data collected for the study will include CPD activity attended, the clinical area of the activity, the type of needs assessment conducted, the theoretical foundation, the length, the number and type of instructional methods used (e.g., group discussion, role-play, video, touch pad) , the number of faculties involved in its design and presentation, and the source of funding. In addition, the content of the CPD activity relating to basic evidence-based information deemed necessary for making quality medical decisions will also be assessed.
In accordance with the integrated conceptual model proposed by Godin and colleagues (2008) to predict behaviour of healthcare professionals, our new global instrument will basically assess participants' information, namely, beliefs about consequences and capabilities, social influences, moral norms, perceptions of role and identity, and their intentions, habits, and past behaviours.
Data collection procedures
Two data collections are planned: (1) a test-retest of the new instrument for assessing its reliability and item analysis and (2) a before-and-after study to test the validity of the instrument.
First, a group of 50 participating physicians (25 family physicians and 25 specialists) who attended an eligible CPD activity will be asked to complete the instrument immediately after the CPD activity and to use the same instrument again after two weeks. The research assistant will offer to mail the instrument to the physicians. This first data collection will provide the data for assessing both intra-subject reliability at two weeks and the internal reliability of the instrument. Reliability of the scales will be assessed with Cronbach's alpha for internal consistency and with an intraclass correlation coefficient for stability. It will thus be possible to modify the instrument based on this first set of results. The final choice of the set of items will be based on their indices of distribution and discrimination and their theoretical importance.
Second, we will use a before-and-after study design. In eligible CPD activities, half of the participants, randomly selected, will be asked to complete the evaluation instrument both before and at the end of the CPD activity, and half of the participants will be asked to complete the evaluation instrument after the CPD activity. Data from the participants who will complete the evaluation instrument before and after the CPD activity will allow us to verify the instrument's capacity to detect differences in constructs of the integrated model. Comparing results from participants who completed the instrument before and after the CPD activity and those who completed it only after the activity will allow us to explore the mere-measurement effect (i.e., if everyone completes the questionnaire twice, it is not possible to ascertain whether the effect can be attributed to the CPD activity or to the action of completing the questionnaire) .
All participants will be contacted three months later by phone in order to collect self-reported clinical behaviour, as targeted in the specific CPD activity. These data will be used to estimate the instrument's effectiveness in predicting behaviour. In accordance with the results of the systematic review of factors predicting behaviour in healthcare professionals , self-reported behaviour will be used as a proxy. Although more robust approaches to measure behaviour, such as chart audit or standardized simulated patients, would be preferable, these methods are not feasible in the context of this project.
For the before-and-after data collection, we will seek the participation of 500 CPD enrollees: 125 family physicians/before and after, 125 family physicians/after only, 125 specialists/before and after, and 125 specialists/after only. Apart from sociodemographic results, the integrated model currently proposes seven essential concepts to predict behaviour in healthcare professionals . Each concept represents a construct. Usually, each construct is assessed with 3 to a maximum of 10 items. A minimum of 100 data entries are needed for an exploratory factor analysis (EFA) . Also, a minimum of 10 data entries are needed per parameter to make estimations for confirmatory factor analysis (CFA) . For example, for a construct that is assessed with 10 items, we will have a minimum of 33 parameters to estimate. Consequently, 430 participants should provide an adequate sample size for performing both EFA and CFA for the two groups together (family physicians and specialists), as well as some of the other planned validity and reliability analyses. We will need 473 physicians to consider a 10% of potential losses to follow-up. Therefore, the recruitment of 500 physicians (250 family physicians and 250 specialists) will provide an adequate power for the analysis.
Using EFA, we will explore the factorial structure within the potential seven constructs describing essential concepts to predict behaviour of healthcare professionals. We will then use CFA to assess construct validity.
We will also assess the relationships between the constructs based on the proposed integrated model using path analysis. Observation of the expected relationships will be further proof of the validity of the model and its constructs. The difference between family physicians and specialists for each construct will be assessed with a multiple-group analysis. Also, in participants who completed the instrument before and after the CPD activity, the effect of participating in a CPD activity and change in the physician's intentions with a paired student t-test will be verified. Our a priori hypothesis is that physician's intentions will be positively associated with participation in the CPD activity. Physician's intentions after the CPD activity calculated in this group will be contrasted with results obtained in the control group to explore the mere-measurement effect . Predictive validity will be estimated by correlating the developed measurement instrument score and ratings from participants on the intention construct with self-reported behaviour at three months. Our a priori hypothesis is that the measurement instrument will correlate in the expected direction with self-reported behaviour as well as with changes in self-reported behaviour.
Phase 4: assessment of the acceptability of the new instrument by decision makers and CPD participants
The objective of phase 4 is to explore the acceptability of the new global instrument by CPD decision makers and gather suggestions of factors that might influence its implementation. Data collection will consist of individual interviews and focus groups with key stakeholders. We will use the contact network of team members to purposefully select participants from within four groups: (1) Québec's key stakeholder CPD organizations, (2) primary care physicians, (3) specialists, and (4) representatives of Québec's Ministry of Health and Social Services. The aim is to highlight anticipated barriers and enablers for implementing the new instrument in CPD practices.
We will collect the participants' sociodemographic characteristics and additional information on their organizations. Interviews and focus groups will be structured according to an interview guide that will facilitate semi-structured discussions regarding the instrument. Examples of questions to be included in the interview guide are as follows: Who do you think might have a favourable/negative attitude to this measurement instrument? Can you think of barriers and enablers for implementing this instrument in CPD practice?
All interviews and focus groups will be tape-recorded and transcribed verbatim. Analysis of the transcripts will occur concurrently with data collection in both the focus groups and the individual interviews and will use a constant comparative method. Briefly, units of data will be categorized and new units compared with previously identified ones in order to develop or saturate each category . NVivo (QSR International, Cambridge, MA, USA) software for qualitative analyses will be used to support data collection, organization, and analysis.
Internal validity of the study will include member checking . A summary of the interpretations of the interviews and the focus groups will be sent to each participant, who will be invited to make comments and corrections. Subsequently, a summary of the barriers and enablers for implementing the measurement instrument in CPD practice and proposed actions to be taken will be circulated and discussed in a conference call with the research team. A report of the results from phase 4 activity will be produced to be used in phase 5.
Phase 5: development of a consensus among CPD decision makers regarding an implementation plan for the new instrument
This research activity will involve meeting with Québec's medical CPD consultative coordinating body. The research methods used in this phase will be based on consensus-building methods, which will include convening, clarifying responsibilities, deliberating, deciding, and implementing agreement. Consensus building is defined as the process of seeking unanimous agreement . The research team will draw on the results from the above-mentioned research activities: (a) the new measurement instrument and its characteristics (e.g., items, validity, and reliability data); (b) a summary of the barriers and enablers as perceived by key stakeholders for implementing this instrument in CPD practice; and (c) proposed actions to assess the impact of CPD activities. Details of the results of these research activities will be sent to team members in preparation for a face-to-face meeting. The face-to-face meeting will be structured around a specific goal: building consensus regarding an implementation plan for the new instrument to assess the impact of CPD activities. The meeting will be led by an external facilitator. The agenda will aim at achieving agreement on a final plan for implementing the new instrument.
Once the instrument has been found to be valid and reliable, we will create an English version using translation and back-translation methods .