Developing the ethics of implementation research in health
- Vijayaprasad Gopichandran†1Email authorView ORCID ID profile,
- Valerie A. Luyckx†2,
- Nikola Biller-Andorno2,
- Amy Fairchild3,
- Jerome Singh4,
- Nhan Tran5,
- Abha Saxena6,
- Pascal Launois7,
- Andreas Reis6,
- Dermot Maher7 and
- Mahnaz Vahedi7
© The Author(s). 2016
Received: 8 June 2016
Accepted: 29 November 2016
Published: 9 December 2016
Implementation research (IR) is growing in recognition as an important generator of practical knowledge that can be translated into health policy. With its aim to answer questions about how to improve access to interventions that have been shown to work but have not reached many of the people who could benefit from them, IR involves a range of particular ethical considerations that have not yet been comprehensively covered in international guidelines on health research ethics. The fundamental ethical principles governing clinical research apply equally in IR, but the application of these principles may differ depending on the IR question, context, and the nature of the proposed intervention. IR questions cover a broad range of topics that focus on improving health system functioning and improving equitable and just access to effective health care interventions. As such, IR designs are flexible and often innovative, and ethical principles cannot simply be extrapolated from their applications in clinical research. Meaningful engagement with all stakeholders including communities and research participants is a fundamental ethical requirement that cuts across all study phases of IR and links most ethical concerns. Careful modification of the informed consent process may be required in IR to permit study of a needed intervention. The risks associated with IR may be difficult to anticipate and may be very context-specific. The benefits of IR may not accrue to the same groups who participate in the research, therefore justifying the risks versus benefits of IR may be ethically challenging. The expectation that knowledge generated through IR should be rapidly translated into health policy and practice necessitates up-front commitments from decision-makers to sustainability and scalability of effective interventions. Greater awareness of the particular ethical implications of the features of IR is urgently needed to facilitate optimal ethical conduct of IR and uniform ethical review.
Implementation research (IR) is growing in importance and recognition: there is an increasing funding from a range of donors/sponsors for this research area, leading scientific journals have established sections promoting the publication of such research, and it contributes increasingly to the evidence-base used by the World Health Organization (WHO), which promotes, supports, publishes, and evaluates such research . With its aim to answer questions about how to improve access to interventions that have been shown to work but have not reached many of the people who could benefit from them, IR involves a range of particular ethical considerations that have not yet been comprehensively covered in most international guidelines on health research ethics [2, 3]. The draft of the Council for International Organizations of Medical Sciences (CIOMS) guidelines which is currently under revision as well as its commentary does briefly allude to ethical considerations in the conduct of cluster randomized trials (CRT), but thus far there has been no comprehensive discussion or guideline regarding the application of ethical principles in IR in general or in relation to study designs beyond CRTs .
In response to the need for more clarity and guidance about the ethical implications throughout the IR process, Special Programme for Research and Training in Tropical Diseases (TDR) and the Global Health Ethics unit in collaboration with the Alliance for Health Policy and Systems Research at the World Health Organization are developing a training tool, the Ethics in Implementation Research Toolkit, as a practical guide for IR researchers and ethics committees to facilitate optimal study design, conduct, and review. The training tool was developed through a consultative process launched in Geneva in which IR experts, philosophers, ethics committee members, and public health practitioners met to identify the ethical issues in IR, define the course content and the format of the training workshops. The tool was further developed through small group work with the support of an expert in adult training methods. The tool has been validated in two pilot workshops in Asia and Africa. The list of experts who contributed to the development of the Toolkit, apart from the authors of this manuscript, can be found in the Acknowledgements section.
This manuscript is constructed around the concepts identified during the development and piloting of the Ethics in Implementation Research Toolkit and aims to highlight the differences in application of ethical principles between clinical and implementation research, and to highlight the current gaps in ethical guidelines for the conduct of IR.
IR involves increasing the understanding of how to improve access to health products and strategies that are already available and have been shown to work, but remain beyond the reach of many of the people who could benefit from them. IR therefore provides the link between what should happen in theory and what actually happens in practice. It is rooted in the identification of practical problems facing disease control programmes and in finding solutions which improve access to health interventions and lead to better health outcomes. IR addresses different aspects of implementation including social and contextual factors (poverty, environment, culture), the process of implementation (which approach best answers the implementation issue?) or the outcomes of implementation (clinical/process end points). For example, in case of a new vaccine for prevention of dengue, basic science and traditional clinical research address vaccine development and safety and efficacy testing. IR then addresses the questions of accessibility, acceptability, appropriateness, and feasibility in the communities where the vaccine is needed. IR questions, however, are not always related to a clinical disease entity or implementation of a treatment or prevention program. IR also addresses process issues in health care delivery, as well as cost-effectiveness, policy uptake and implementation, health education etc. IR therefore draws on a wide variety of research approaches to address the diverse research questions. The research designs therefore are not restricted to traditional trial designs, but include methods such as participatory action research (PAR),1 qualitative design, and effectiveness implementation hybrid designs 2. Flexibility is a great advantage in IR as the research question largely drives the design, the research process is iterative, and the findings at each stage feed back into the design. IR is usually carried out in close collaboration between researchers and disease control programme staff or policy-makers. The costs are generally modest, yet IR has the potential for a large magnifier effect, as effective implementation expands the impact of health interventions delivered by programmes. As a pre-requisite therefore to the design of a successful implementation strategy, the clinical/public health problem must be identified, the epidemiology of the disease/health status must be understood, and a situation analysis must be performed to identify why access is sub-optimal, and what the actual bottlenecks/gaps in care delivery are (not merely presumed), such that interventions can be targeted to reduce these bottlenecks/gaps. In the case of adoption of a successful intervention from one country by another country or scaling-up of interventions from a pilot phase to a wide area, a local situational analysis should be carried out to determine differences and similarities between the communities where an intervention has been successfully implemented and the communities in which the intervention will be tested. IR is relevant when this analysis shows important differences but points to the proposed intervention as the most appropriate strategy, or justifies full scale implementation of the intervention . Awareness of the appropriate application of ethical principles in IR is important in study design and data generation to ensure ethical conduct of IR and to effectively contribute to health system strengthening. In the planning stages, researchers must also be able to effectively communicate their consideration of the ethical principles to research ethics committees, who must also have insight into the adaptations of ethical principles required in IR (as opposed to traditional clinical research) such that protocols are appropriately and fairly reviewed.
Are there ethical considerations which apply particularly to implementation research?
Differences between clinical and implementation research which impact application of ethical principlesa
Countries, institutions, communities, and individuals
Informed consent by competent individuals, assent by minors and consent by legally authorized representatives
Consent may be difficult to obtain in cluster randomized trial design. There may be a need for a two level consent—consent for randomization from gatekeepers and consent for participation at the individual level. Sometimes individual consent may not be feasible. However, gatekeeper consent does not replace the need for individual consent. Ethical committee should oversee the informed consent requirement and process
Clinical as well as contextual equipoise (genuine uncertainty that the implementation will work in a new context as well as whether the implementation package will work at all)
Understanding of disease pathophysiology
Intervention aimed at disease-specific management
Identification of population health needs
Understanding relative priority of need for intervention within local context
Community engagement to understand community needs, ensure scalability, and sustainability
Generally controlled research environment
Real-life or pragmatic research environment
Cross-sectional, case-control studies,
Cohort studies, randomized clinical trials
Cluster randomized trials
Pragmatic, mixed methods, effectiveness implementation hybrid designs, participatory action research, quasi-experimental design, realist review
Integration within health system
Often, there is no a priori plan for health system integration. Findings of clinical research go through IR before integration into health system
IR has a strong health system strengthening focus. It creates horizontal integration into the health system. There is an ethical imperative for health system integration
Predominant research disciplines
Physiology, genetics, biochemistry, and other basic sciences, epidemiology, clinical medicine
In most epidemiological designs, control groups are required. But some phase 1 clinical trials and observational studies may not require control groups
Having a no intervention control group may not be acceptable. Alternative designs of quasi-experimental studies do not require a control group
Boundary between research and clinical care
This boundary is usually clear, but may be unclear in case of therapeutic misconception especially in cancer trials
Is often unclear, because the intervention is of proven efficacy
Types of research question
Efficacy and safety of a therapeutic strategy in the individual
Operationalization of an intervention in local context
Implementation of an intervention in local context prior to scale-up
Health system functioning at multiple levels
Well-defined hypothesis at the beginning of the clinical research. Expected outcomes clearly stated.
Multifaceted holistic impact on health systems functioning with regard to intervention tested. Sometimes outcomes may be unexpected
Risks assumed by:
Mostly, the risks are for the study participants. However, families and communities may also be affected in specific contexts
Usually population level risks. Moreover, the people getting the benefits and people suffering the risks may be different.
Benefits accrued by:
Benefits accrue to the participants, the community. The research finding may be a common good
Individuals, communities, health system, institutions may benefit. The research findings may be common good. The people accruing benefits may be different from those who suffer risks
Generalizability is sometimes possible in multicentric and well sampled studies, however most studies are specific to the target populations.
Generalizability may be limited by contextual factors. However, findings may be generalizable to similar contexts
Social justice implications
Social justice is usually not a primary consideration. However, justice considerations are required in selection of research participants. Research on vulnerable participants is often contentious because of compromised autonomy and other logistics
Social justice considerations are primary. Working with vulnerable groups essential to understand implementation issues in these groups so that the intervention can reach them
A case study illustrating the multiple ethical challenges arising in implementation research
Implementation research of strategies to improve vaccine coverage in children in nomadic populations
Study description. Country X had a low rate of vaccine coverage largely because a significant group of nomadic populations were not reached by the routine vaccination strategies. A basic needs assessment was performed among the nomadic populations and found their vaccine coverage rate to be very low. In addition, the assessment found that insufficient knowledge of the location of the nomadic populations, lack of logistical support and lack of community engagement in the vaccination drives were important reasons for poor coverage. In order to overcome these problems, an implementation research study was planned. Special Outreach Teams (SOTs) were trained and deployed to a selected sample of known nomadic groups. These SOTs were provided with all logistical requirements such as vehicles, ice boxes, vaccine stocks, and temperature logs. They were also trained to engage with the communities, to deliver the vaccinations to the children under five according to schedule and also to collect data on the existing level of vaccine coverage, numbers vaccinated, documentation of feasibility challenges, and costs. The SOTs coordinated their work with the routine health care workers in the communities where these nomadic groups were stationed at the time of contact. In addition, a small subsample of the nomadic groups were invited to participate in a mobile phone-based GPS tracking study to assess the feasibility and utility of locating the nomadic groups in real-time. Solar powered battery packs were provided to the key members of the community who held the GPS tracking mobile phone. Their GPS coordinates were relayed to the SOTs so that they can deliver their services effectively.
Ethical issues. This implementation research study brings out several important ethical considerations. There is an ethical imperative to engage and work with this special marginalized group in order to increase coverage of vaccination as it is the duty of the health system to protect and promote their health and also in order to more effectively protect the rest of the community with whom the nomadic groups will come in contact. Equipoise to justify the conduct of the study lies primarily in the uncertainty of how the intervention will be taken up and effective within this context as it is known that the vaccinations are effective from other populations. Despite this fact, however, effectiveness of the vaccine should be tracked in this new population as there may be modifiers of the effect, e.g., nutritional status etc. that may also require ancillary care. The findings of the study will help understand the feasibility and acceptability of the intervention among nomadic communities thus facilitating the implementation of the vaccination coverage campaign. Community engagement is a key ethical consideration in this context. Marginalized communities like these have inherent mistrust in health systems and community engagement helps build trust. Identification of appropriate gatekeepers of the community by appropriate selection process will facilitate both the informed consent process as well as representation of long-term voices of the communities. The tracking of the position of the nomadic groups for the sake of facilitating the vaccination process using the GPS tracking system is a significant intrusion into the privacy of the communities. This needs to be carefully weighed and balanced against the benefits of enhanced vaccine coverage and reduced child mortality because of that. Moreover, confidentiality of the GPS tracking data should be clearly maintained. Issues of sharing the GPS tracking position with local health system, other parties who may be interested to track them for other purposes etc. needs to be carefully deliberated. In addition, it is likely that many other health needs would be identified in these communities, raising the ethical issues associated with ancillary care responsibilities.
Ethical concerns in the planning phase of IR
Responsiveness of IR
Problems addressed by IR must be of high local priority in order to justify the research . One could argue that the requirement for responsiveness is greater from IR compared with clinical research as IR should address priority health needs whereas clinical research is concerned more with proof of principle, and wider application should be tested with subsequent IR. Conversely, when there is an available intervention for an important unmet health need in a community, there is an ethical imperative to conduct IR to try to address the implementation barriers. As such, IR studies are often commissioned by local health authorities. The knowledge of which problems are indeed local priorities relies on epidemiology and health data reporting, which may not always be optimal in resource-challenged environments. Engagement with local health experts and communities is therefore essential in the planning stages of IR to determine whether a health problem is indeed perceived to be a local priority. If a particular problem is not perceived to be a local health priority, the ethics of conducting IR becomes questionable .
Equipoise is an important ethical imperative in the conduct of research, it is required to justify any potential risk to research subjects. Clinical equipoise refers to the fact that investigators conducting a randomized controlled trial do not know in advance if an intervention is better than what it is being compared with. In IR, however, such clinical equipoise is generally not present (e.g., a medication is known to cure malaria), but instead, situational or contextual equipoise justifies the conduct of IR, i.e., there remains genuine doubt whether a new and untested package of interventions will work in a specific context . To ethically justify IR, therefore, equipoise regarding the effectiveness of the implementation processes must be preserved.
Ethical issues relating to examples of implementation research designsa
Cluster randomized trials (group randomized, place-based, community wide intervention trials)
-Random allocation of groups or “clusters” to study arms and outcomes are measured in individual subjects and at community level
-Randomization of clusters of obstetrics unit staff to education on hand washing or usual practice, measurement of rates of puerperal sepsis in women delivering at study clinics
-Different units of intervention and outcomes measurement
-Consent before and after randomization, whom to consent?
-Choice of gatekeepers
-No opt-out option within cluster
-Risk: benefit balance
-Ethics of randomization to known intervention, equipoise,
-Identification of vulnerable groups
Effectiveness-implementation hybrid trials
-Assess both effectiveness and implementation strategy simultaneously
-Identify intervention—implementation interactions
-Evaluate impact of ITN on reduction of malaria and assess robustness of availability and uptake of ITNs in the community
-The trade-off between the scientific rigor required for effectiveness assessment and the realistic contextual considerations required for implementation is an important ethical consideration
-Use of both qualitative and quantitative methods
-Understands various perspectives
-Rationales: “participant enrichment”, “instrument validity”, implementation validity”, “meaning enhancement”
-Integration of HIV and TB management in single clinics—patient experience (qualitative) and adherence (quantitative)
-The trade-off between the scientific rigor required for quantitative methods and the realistic contextual considerations required for the qualitative component
Participatory action research
-Research question, design, and data collection in a participative manner by the research participants
-Peer support groups to improve adherence to ARV in HIV + subjects
-There is a need for community engagement to ensure responsiveness, sustainability, and scalability
-Effects of intervention in routine practice
-Maximize variability of settings, practitioners, patients
-Introduction of community health workers for home management of malaria
-There may be concerns of standards of care and ancillary care, which in pragmatic conditions may be ethically debatable.
-With or without control group
-Open label demonstration project of effectiveness of self-reported use of pre-exposure prophylaxis for HIV
-There is a concern regarding scientific rigor of the research
-Analysis of how and why an intervention works in a context combining theory and empirical evidence.
-Integration of traditional healers into home management of malaria strategies
-Community engagement is of utmost importance to retain cultural and contextual sensitivity
Stakeholder and community engagement
The term stakeholder has numerous definitions, many of which are contextual . Two general definitions are relevant in the context of IR: the first defines a stakeholder as a “person or group with an interest, involvement or investment in something” ; the second describes stakeholders as “people who will be affected by a project, or who can influence it, but who are not directly involved in doing the work” . Key stakeholders in IR may include the government, policy-makers, public health functionaries, health care providers, health care managers, financing mechanisms, health care industry, and the community. Communities may include individuals on who interventions are planned, the broader community or social structures to which these individuals belong and the broader society to whom an intervention may eventually be rolled out. Communities and individuals with specific roles are important stakeholders in the research process. Meaningful engagement with stakeholders at all levels is crucial in IR, as a means to identify health priorities, to identify key participants, to communicate transparently and effectively about the goals, design, risks, benefits, and process of a proposed intervention, to gain trust and develop partnerships to enhance success of the study, and to gain feedback and identify unforeseen barriers that could be mitigated at the planning stage [31, 32]. Community engagement is a related but different concept where the members of the community who will benefit from or face the risk of the IR are actively consulted and engaged with, with the goal that they play an active partnership role throughout the IR process.
Stakeholder and community engagement are cross-cutting processes which must be carried out during the planning, implementation, and post-research phases of IR. Particular ethical underpinnings of engagement with policy-makers and health financers at the planning stage include determination that the intervention will address a local priority health need and to gain buy-in and commitment for the scale-up and sustainability of an effective intervention. Scale-up is considered by some to be an ethical corollary of IR. Important ethical goals for engagement with communities as partners in planning and design of IR include to determine acceptability of a proposed intervention, maximize uptake, ensure inclusion of vulnerable populations, establish accountability processes, and particularly when individual informed consent is not feasible due to the research design, to ensure individuals are aware of the rationale and opt-out possibilities. The process of decision-making in IR should consider the power differential between the researchers and the community and allow adequate representation of the research participants and the community at large [33–35].
Often community representatives are selected to facilitate communication between researchers and the community. The selection of community representatives must be an inclusive and fair process of democratic election or nomination, guided by the community itself, to ensure appropriate, acceptable, and comprehensive representation of all sectors of the community irrespective of class, race, gender, sexual orientation, or ethnicity and to avoid any potential conflicts of interest-specific individuals may have [33–36]. Engagement with disadvantaged and marginalized groups is imperative up-front and throughout the IR process to ensure acceptability and equitable participation in IR, and importantly, to identify any specific unanticipated barriers they may face and to develop strategies to mitigate further marginalization or stigmatization through the research [8, 37].
The balance between the risks and benefits
The clinical efficacy and safety of an intervention is generally known before IR is conducted. In clinical research, an individual participant can personally weigh the relative risks and benefits before giving their informed consent, and the risks and benefits are usually borne by the same individual. For example, in clinical research testing, the efficacy of a new vaccine, the benefit of personal protection, and the risk of side-effects are borne by the individuals who participate in the study. In IR, for example in mass drug administration interventions, the community may benefit from large-scale treatment of individuals, but an individual may experience side-effects from a medication they may not personally have required. In addition, the potential risks of an IR intervention may also result from the modality of implementation . For example, a community wide public health screening campaign for sexually transmitted infections which had been successful in one low-income country may carry different risks of stigmatization, religious ostracism, and social discrimination if implemented in an underdeveloped and religiously orthodox country, leading to a different risk-benefit balance. The risks associated with IR may not always be obvious up-front as health systems are complex adaptive systems, and interactions between the components in the health system are not often clearly understood . Diligent situational analysis must therefore be conducted during the planning phase of IR to identify potential risks before harm is done . In addition, a particular feature of IR is that at times an intervention is implemented in one group, but the benefit may accrue to another group . For example, IR studying the implementation and uptake prior to scale-up of a malaria transmission-blocking vaccine exposes vaccinated individuals to the risks of vaccination, but unless a large proportion of the community is vaccinated, the individuals vaccinated will protect others from malaria transmission, but will not be protected themselves. How to balance the risks experienced by one group against the benefits gained by another requires ethical deliberation and effective communication with the research participants. The ethical deliberation should be based on the solidarity principle and should be transparent, involving communities and all stakeholders . To what extent individuals within a group should be exposed to risks for the benefit of others cannot be clearly defined, but it should be decided based on community and stakeholder consultations . A line which cannot be crossed is knowingly exposing one group to harm or significant risk for the benefit of another.
Ethical concerns in the implementation phase of IR
Autonomy and informed consent
A key principle driving the ethics of clinical research in humans is individual autonomy. In public health research autonomy has two dimensions, one concerns individual autonomy and the other concerns relational autonomy in the context of the community to which the individual and the health system belong . Informed consent is the process through which a research participant can exercise their autonomy. In clinical research, a fully informed individual can determine whether or not they wish to freely participate in a study and can usually opt-out of the research at any stage. In IR, there may be difficulties in operationalizing informed consent [2, 20, 41, 42]. For example, an individual in a cluster in a CRT may not have the chance to decide and give consent to randomization as randomization happens at the cluster level. In the case of non-excludable cluster level, interventions such as environmental modifications, an individual may not be able to exercise a meaningful refusal to participate. In such situations in IR therefore there is a need to articulate informed consent differently from traditional individual consent in a clinical trial. At one end of the spectrum is a complete waiver of individual informed consent where the ethical risks are minimal, and the interventions are largely at a cluster level (i.e., no individual can opt-out), rendering refusal meaningless. For example, in an IR study of an ultraviolet wave system to provide safe drinking water to a population, the harms are considered minimal, and it is not possible for any individual participant in a cluster randomized to the intervention arm to easily opt-out of the study . Such a waiver of consent does not however preclude the need for meaningful community engagement and provision of information. At the other end of the spectrum is the example where individuals have the opportunity to refuse participation in the research project, even though the intervention will occur at the cluster level. This could occur for example when public health professionals wish to test whether community health workers can be trained to provide injections in the community, and individuals have the option to refuse participation and visit the health facility to receive the injection from a nurse. Individual informed consent would be the norm in this case, even though consent for randomization cannot be provided by the individual and it is operationalized at the cluster level prior to individual contact [20, 43]. In the middle of the spectrum is dual consent from gatekeepers and individuals. Community agreement relies on the identification of an appropriate gatekeeper, who should have a keen interest in the welfare of the community and represents the community in a fair manner . There are several challenges in selection of the gatekeeper. In traditional communities where collective decision-making is practiced, selection of a gatekeeper may not be problematic. But in more complex societies, or more complex studies, selection of one voice to represent the community is often challenging. The community leader may not be the most appropriate person to make decisions on whether a community or its members should participate in a study or not. For example, an elderly male village leader may not be an appropriate gatekeeper to consent to an IR intervention on pregnant women in his community. Selecting community representatives fairly requires inclusion of a variety of representative stakeholders, especially those from the target groups, and ensuring transparency of the process [20, 36]. The agreement of the gatekeeper, however, cannot replace individual consent or assent where relevant as discussed above . Ultimately, it is important that proper ethical oversight is in place through Institutional Ethics Committees to ensure that the appropriate informed consent process is followed, maximally respecting autonomy of individuals in the study .
Challenges in operationalizing informed consent in the context of IR also include whether the beneficiaries are individuals or populations, and appropriate identification of who the actual research participants are . For example, when implementing a taxi voucher system to increase the rates of institutional deliveries and reduce maternal mortality, should consent be obtained from the pregnant women, the taxi drivers, or the health care workers whose performance will also be evaluated? The Ottawa statement on ethical conduct of cluster randomized controlled trials define a research participant as: the intended recipients of the experimental or control intervention; the direct targets of experimental or control environmental alterations; persons with whom researchers interact to collect their data; persons whose identifiable private information is accessible to the researcher for collection of their data . As such, in the example above, patients and health care workers should provide consent, but whether this should extend to the taxi drivers is questionable and may be difficult to operationalize. A further important ethical issue in the informed consent process is the extent of information to be revealed to the participants in the intervention and control arms, where applicable. In IR, especially when behaviour change interventions are being studied, knowledge of the intervention itself may change the outcomes and implementation process. There is, therefore, often the need to conceal some information about the intervention. The ethical justification for this is debatable, and it must be balanced against the risks/benefits and the potential impact on study validity as discussed in the CIOMS guidelines . The informed consent process in IR therefore may be quite different from that in clinical research and requires thorough consideration to ensure optimal ethical conduct of IR.
Privacy and confidentiality
Particular issues relating to privacy and confidentiality in IR relate to the fact that IR often requires that facility level data on patient outcomes be available or that individual level data from facility health records be obtained. For example, if a public health intervention is implemented to regulate institutional deliveries and improve the quality of skilled institutional deliveries, there may be interventions at the health facility level, but confidentiality restrictions on access to data from women who deliver in the facilities may hamper effectiveness analyses of the health system impact . In such cases, the data that is obtained from facilities should either be anonymized, or the individuals about whom data is being sought should provide consent for their data to be reviewed by the researchers. Where such consent is not possible, it is the responsibility of the researcher to obtain a waiver of consent from the respective ethics board, and put in place mechanisms to ensure that the confidentiality of the patient information is respected. A proactive strategy of informing patients about potential data collection for research and quality improvement purposes up-front, but reassuring them about privacy and confidentiality could also serve strengthen the patient-researcher partnership and build trust .
Standard of care or prevention
There are two approaches to decide on standard of care or prevention to be given to a control group [47–49]. One approach is to allocate the local de facto existing standard, which in some situations may be grossly insufficient, making it ethically unacceptable based on justice and fairness principles. For example, in an IR trying to study prevention of mother to child transmission of HIV in a country where routine anti retroviral therapy is not available, the local de facto standard of care is no treatment. Having a placebo control arm in the study is not acceptable in spite of the local de facto care being no treatment, because an effective treatment which reduces transmission is available and should be accessible to the mothers. The second approach is to provide the local de jure standard of care or prevention, which is agreed upon by public health experts of that region and is acceptable to the community. This approach may still be unfair in that this standard may be unsustainable for the local health system after the IR is completed. For example, in a public health behaviour change implementation study focused on hand washing among schoolchildren, the intervention group receives a school-based lunchtime hand washing program, and the control group receives soap and water in all schools, but without any emphasis on hand washing before eating. In this case, the standard of care is provision of hygiene tools, and the intervention is emphasizing the use of these tools. In this context, allowing the control group to have no intervention can be considered ethical. The consideration of standards of care or prevention may therefore identify new gaps as targets for future IR.
Ancillary care refers to the identification of problems that may contribute to ill-health that are beyond the scope of the study in question, for example, researchers studying home management of malaria may come across household members with other diseases needing attention [11, 50, 51]. Sometimes ancillary care responsibilities can be foreseen at the design stage and at others they are encountered only during the conduct of the IR. Ancillary care obligations are present when the need is serious in terms of severity or urgency or both and when there is a possibility of provision of care within the scope of the research . For example, in the school-based hand washing behaviour change IR study, uninterrupted tap water supply may be lacking and this is an ancillary care requirement. However, this example illustrates that it may not be realistic to expect implementation researchers to assume all ancillary care responsibilities. Researchers may not have the expertise to provide the ancillary care; the provision of the care may be costly or may require system-level interventions. The researchers must, however, establish process of accountability for ancillary care need identified through the research, determine which needs may realistically fall within the scope of responsibility of the researchers, and proactively engage with the local government or non-governmental organizations during the planning and conduct phases of IR to identify who will be able to meet other needs .
Research capacity and health system strengthening
Well conducted IR should lead to strengthening of research capacity of the local institutions as well as individuals’ capacity to conduct research in settings where such capacity is weak . Research capacity strengthening can range from creating a trained workforce of researchers and research volunteers up to training and capacity building of research experts and infrastructure to permit independent conduct of locally responsive IR in the future. Based on the need in the area where the IR is being conducted, appropriate research capacity strengthening should be facilitated. This can be facilitated by appropriate stakeholder engagement ensuring commitment by donors and governments to build sustainable research capacity. Not only is it important for the IR to strengthen local research capacity, it should also strengthen the health system within which it is conducted. For example, true partnership in an implementation research study of rapid diagnosis of tuberculosis resistance should build sustainable infrastructure in technology required, expertise to run and maintain the technology, strengthen the local health information system to track data acquired, train local researchers in design, conduct, analysis and reporting of study findings as well as participation in post-intervention scale-up, thereby strengthening the local research and health system capacity. IR projects focusing on specific health gaps may, however, create vertical program structures within the health system which may be disempowering to the system through inefficient resource utilization . It is ethically important that the conduct of IR should focus on horizontal integration of public health interventions into the health system such that a project empowering any component of the health system may have positive repercussions for the entire system. Strengthening the capacity to translate research findings into health policy is a specific imperative in IR and must be a component of all phases of the IR process .
Ethical concerns in the post-research phase of implementation research
Dissemination of research findings
Given the important public health impact of IR, there is an ethical obligation to disseminate the research findings widely, including feeding back to the communities and stakeholders who participated in the research [54, 55]. If an implementation strategy had a negative or positive impact in a certain context, either finding may be important for researchers planning similar interventions elsewhere. Therefore, irrespective of the results, the findings of the IR should be disseminated. Furthermore, resource utilization globally could be enhanced by an imperative for dissemination of IR findings, as once an intervention has been tested in many different local contexts, its findings may be presumed to be generalizable and obviate the need for new IR studies and delays in scaling-up of the intervention in new contexts.
Data ownership and sharing
In case of donor or sponsor-driven IR, data is often owned donor, who may regulate and restrict further handling of the data. Data ownership should be fairly negotiated through transparent stakeholder engagement in the planning phase of IR, and ethical oversight of the data ownership process is required to ensure appropriate access to the research findings by the relevant stakeholders post-study, including the local researchers and communities when appropriate, to maximize the utility of the knowledge generated. It may be acceptable for researchers or donors to own data without further responsibility. However, given the policy and public health implications of IR and the necessity of trust especially with the communities, there may be a responsibility for data sharing which should also be negotiated up-front, considering the important implications of protecting privacy and confidentiality as well as to allow strengthening of local research capacity.
Translating findings into public health action
Due to the inherent nature of IR, there is an ethical obligation for IR findings to be used to inform effective and equitable public health action. This necessitates timely consideration and uptake as relevant of the IR findings into public health policy and practice. Potential barriers to translating knowledge into action include lack of prior consultation with policy-makers, lack of funding, weak health systems, poor communication of findings by researchers to policy-makers, and absence of a culture of evidence-based decision-making among others . Therefore, in order to translate the research into public health action, implementation researchers should engage with policy-makers and health system officials, important stakeholders in IR, upfront to ensure commitment to sustainability should the intervention prove successful, and must communicate their research findings rapidly, clearly, and concisely to engage and inform policy-makers in a timely fashion. Researchers should also propose actionable suggestions based on the research findings to facilitate uptake and scale-up of successful interventions. Barriers identified during IR may require further study to develop strategies to overcome them. Effective communication between researchers and policy-makers, as well as education of the public are important social justice obligations in IR, ensuring that expectations raised during the research are met, and those who participated as control subjects gain access to interventions withheld from them during the study.
Scalability and sustainability
Scalability and sustainability are important ethical considerations at both planning and post-study phases, as ultimately these are the goals of IR [13, 56]. The duty to ensure sustainability post-study cannot only lie with the researchers. Multiple stakeholders must come together to promote this goal which requires ongoing stakeholder engagement throughout the IR process. The researchers should ensure through effective engagement during the planning and conduct stages of IR that the non-research stakeholders such as policy-makers, local providers, and health system officials remain committed to sustaining implementation of an intervention if found to be effective. If access to a proposed public health intervention cannot be ensured for a community after the IR, it may not be ethical to carry out such a research activity. If specific interventions are provided during IR without a plan for sustainability, this could lead to exacerbation of inequity and harmful effects to the community as well as loss of trust in the health system.
Irrespective of the context in which IR is conducted, LMIC or developed countries, there is an ethical obligation to share benefits of the IR with the community . There are various classifications of the benefits that can be achieved as a result of conduct of the IR. The benefits may be direct as a result of the intervention being studied, or indirect and not related to the intervention per se. The benefits may accrue to individual participants or to the community at large. For example, IR may be conducted in communities where the local health system is weak, therefore success of an intervention may result in introduction of a new intervention that was effective in the local context, providing individual benefit. In addition, the IR likely identified and overcame barriers which would have contributed to some strengthening of the local health system that would have a broader impact. IR researchers can facilitate sharing of direct benefits by advocating for sustainable translation of research findings into action, and sharing of indirect benefits through building research capacity and health system strengthening. The unique nature of IR where the individuals who bear the risks of the IR are not always the ones who enjoy the benefits is a challenge as discussed above. Optimal benefit sharing can be promoted through proper pre-IR planning and ethical conduct. Community and stakeholder engagement plays an important role in achieving benefit sharing as when adequately informed they can advocate for access to proven benefits. Benefit sharing has important social justice implications, and it is the obligation of the researcher to achieve a balance of risks and benefits to both individuals and communities [58, 59].
Putting public health evidence into practice in specific populations requires the generation of knowledge about the feasibility of public health interventions within a specific context, the relative harms and benefits, how an intervention is taken up, whether it reaches the most vulnerable populations, and the logistics of the implementation process. IR aims to generate this knowledge with the goal of enhancing health system performance while upholding fairness and justice in the reach of the intervention to all parts of the community. The ethical principles pertaining to IR are not unique to IR, but may require adaptation in application given the particularities of IR. The stakes of IR are high because of the research contexts within fragile health systems, the large numbers of subjects involved and the reduced ability to predict outcomes and consequences as compared to clinical research . Awareness of the ethical challenges relating to IR is important throughout the planning, implementation, and post-study phases of the research not only to ensure studies are conducted appropriately and that results are maximally useful, but is also important for ethics review committees and institutional review boards to provide appropriate and insightful review of IR projects.
This paper emerged out of the development of the Ethics in Implementation Research Toolkit. Through the consultation process, an important need was identified to clarify the differences in the application of research ethical principles between clinical research and IR, both to guide researchers in planning and conduct of IR and to facilitate review of IR proposals by research ethics committees. As such, this paper complements the Implementation Research Toolkit (http://www.who.int/tdr/publications/topics/ir-toolkit/en/) and the Framework for Operations and Implementation Research in health and Disease Control Programs (http://www.who.int/hiv/pub/operational/or_framework.pdf). It is hoped that this paper will generate discussion in further refining roles and obligations of implementation researchers in low resources settings and in further defining the obligations of policy-makers and funders in committing to long-term sustainability of successful interventions.
PAR is a community-based approach to research that emphasizes community participation in research design, data collection, analysis, and interpretation. PAR tries to understand situations and phenomena by changing them and making observations. The experimentation is grounded in community wisdom and experience.
Effectiveness implementation hybrid designs include simultaneous testing of clinical effectiveness and effectiveness of the method of implementation. It may be of three types: (1) primarily doing an effectiveness study while at the same time gathering data on implementation, (2) primarily doing an implementation study while simultaneously gather data on effectiveness, or (3) dual testing of both effectiveness and implementation strategies.
The authors acknowledge the participation of the following experts in the first consultation meeting and their contributions: Angus Dawson, Georg Marckmann, Ron Bayer, Clement Adebamowo, Michael Selgelid, Florencia Luna, Margaret Gyapong, Veronica Talo, Bagher Larijani, Drue Barrett, Varalakshmi Elango (adult learning expert).
This manuscript did not receive funding from any source.
Availability of data and materials
This manuscript does not report findings from any particular data. Therefore, data availability is not relevant in this case.
VG and VAL conceptualized and drafted the paper. NBA, AF, JS, AS, PL, AR, and MV contributed to various parts of the manuscript and reviewed several drafts. NT and DM gave critical inputs and reviewed later drafts of the manuscript. All authors have read and agreed to the contents of the final draft of the manuscript.
The authors declare that they have no competing interests
Consent for publication
This manuscript does not report any individual person’s data. Therefore no consent for publication is applicable.
Ethics approval and consent to participate
This manuscript does not report findings of research on human or animal participants. Therefore no research ethics approval was obtained.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Hales S, Lesher-Trevino A, Ford N, Maher D, Ramsay A, Tran N. Reporting guidelines for implementation and operational research. Bull World Health Organ. 2016;94(1):58–64.View ArticlePubMedGoogle Scholar
- Hyder AA, Rattani A, Krubiner C, Bachani AM, Tran NT. Ethical review of health systems research in low- and middle-income countries: a conceptual exploration. Am J Bioeth. 2014;14(2):28–37.View ArticlePubMedGoogle Scholar
- Mills A. Health policy and systems research: defining the terrain; identifying the methods. Health Policy Plan. 2012;27(1):1–7.View ArticlePubMedGoogle Scholar
- Guidelines Revision of CIOMS 2002 International Ethical Guidelines for Biomedical Research Involving Human Subjects [http://www.cioms.ch/final_draft_CIOMS_guidelines-10_september_2015-WITH_WATERMARKS.pdf] Accessed 7 Dec 2016.
- Peters DH, Adam T, Alonge O, Agyepong IA, Tran N. Implementation research: what it is and how to do it. BMJ. 2013;347:f6753.View ArticlePubMedGoogle Scholar
- Bamford R. Ethical review of health systems research: vulnerability and the need for philosophy in research ethics. Am J Bioeth. 2014;14(2):38–9.View ArticlePubMedGoogle Scholar
- Dereli T, Coskun Y, Kolker E, Guner O, Agirbasli M, Ozdemir V. Big data and ethics review for health systems research in LMICs: understanding risk, uncertainty and ignorance -- and catching the black swans? Am J Bioeth. 2014;14(2):48–50.View ArticlePubMedGoogle Scholar
- Rennie S. Tinkering with the health of the poor. Am J Bioeth. 2014;14(2):43–4.View ArticlePubMedGoogle Scholar
- Gupta S. Ethical review of health systems research in low- and middle-income countries: research-treatment distinction and intercultural issues. Am J Bioeth. 2014;14(2):44–6.View ArticlePubMedGoogle Scholar
- Hurst SA. Simplicity as progress: implications for fairness in research with human participants. Am J Bioeth. 2014;14(2):40–1.View ArticlePubMedGoogle Scholar
- Olson NW. Conceptualizing ancillary care obligations in health systems research. Am J Bioeth. 2014;14(2):46–7.View ArticlePubMedGoogle Scholar
- Daniels N. Toward ethical review of health system transformation. Am J Public Health. 2006;96(3):447–51.View ArticlePubMedPubMed CentralGoogle Scholar
- Hyder AA, Pratt B, Ali J, Kass N, Sewankambo N. The ethics of health systems research in low- and middle-income countries: a call to action. Global Public Health. 2014;9(9):1008–22.View ArticlePubMedPubMed CentralGoogle Scholar
- Rennie S, Behets F. AIDS care and treatment in Sub-Saharan Africa: implementation ethics. Hast Cent Rep. 2006;36(3):23–31.View ArticleGoogle Scholar
- Gidado SO, Ohuabunwo C, Nguku PM, Ogbuanu IU, Waziri NE, Biya O, Wiesen ES, Mba-Jonas A, Vertefeuille J, Oyemakinde A, et al. Outreach to underserved communities in northern Nigeria, 2012-2013. J Infect Dis. 2014;210 Suppl 1:S118–24.View ArticlePubMedGoogle Scholar
- Pratt B, Hyder AA: Reinterpreting responsiveness for health systems research in low and middle-income countries. Bioethics 2015;29(6):379-88.Google Scholar
- Peters DH, Tran NT, Taghreed A: implementation research in health: a practical guide. In. Edited by Research AfHSaP. Geneva, Switzerland: World Health Organization; 2013.Google Scholar
- Macklin R. Ethical challenges in implementation research. Public Health Ethics. 2014;7(1):86–93.View ArticleGoogle Scholar
- Chaney E, Rabuck LG, Uman J, Mittman DC, Simons C, Simon BF, Ritchie M, Cody M, Rubenstein LV. Human subjects protection issues in QUERI implementation research: QUERI Series. Implement Sci. 2008;3:10.View ArticlePubMedPubMed CentralGoogle Scholar
- Osrin D, Azad K, Fernandez A, Manandhar DS, Mwansambo CW, Tripathy P, Costello AM. Ethical challenges in cluster randomized controlled trials: experiences from public health interventions in Africa and Asia. Bull World Health Organ. 2009;87(10):772–9.View ArticlePubMedPubMed CentralGoogle Scholar
- Weijer C, Grimshaw JM, Eccles MP, McRae AD, White A, Brehaut JC, Taljaard M, Ottawa Ethics of Cluster Randomized Trials Consensus G. The Ottawa statement on the ethical design and conduct of cluster randomized trials. PLoS Med. 2012;9(11):e1001346.View ArticlePubMedPubMed CentralGoogle Scholar
- Brown CA, Lilford RJ. The stepped wedge trial design: a systematic review. BMC Med Res Methodol. 2006;6:54.View ArticlePubMedPubMed CentralGoogle Scholar
- Binik A, Weijer C, McRae AD, Grimshaw JM, Boruch R, Brehaut JC, Donner A, Eccles MP, Saginur R, Taljaard M, et al. Does clinical equipoise apply to cluster randomized trials in health research? Trials. 2011;12:118.View ArticlePubMedPubMed CentralGoogle Scholar
- Harris AD, McGregor JC, Perencevich EN, Furuno JP, Zhu J, Peterson DE, Finkelstein J. The use and interpretation of quasi-experimental studies in medical informatics. J Am Med Inform Assoc. 2006;13(1):16–23.View ArticlePubMedPubMed CentralGoogle Scholar
- Palinkas LA, Aarons GA, Horwitz S, Chamberlain P, Hurlburt M, Landsverk J. Mixed method designs in implementation research. Adm Policy Ment Health. 2011;38(1):44–53.View ArticlePubMedGoogle Scholar
- Patsopoulos NA. A pragmatic view on pragmatic trials. Dialogues Clin Neurosci. 2011;13(2):217–24.PubMedPubMed CentralGoogle Scholar
- Amico KR, Mehrotra M, Avelino-Silva VI, McMahan V, Veloso VG, Anderson P, Guanira J, Grant R, iPrEx Study T: Self-reported recent PrEP dosing and drug detection in an open label PrEP study. AIDS Behav 2016;20(7):1535-40.Google Scholar
- Stakeholder Involvement. Background paper prepared for the WHO/WEF joint event on preventing noncommunicable diseases in the workplace (Dalian/ China, September 2007).. [http://www.who.int/dietphysicalactivity/griffiths-stakeholder-involvement.pdf]. Accessed 7 Dec 2016.
- GBDRF Collaborators, Forouzanfar MH, Alexander L, Anderson HR, Bachman VF, Biryukov S, Brauer M, Burnett R, Casey D, Coates MM, et al. Global, regional, and national comparative risk assessment of 79 behavioural, environmental and occupational, and metabolic risks or clusters of risks in 188 countries, 1990-2013: a systematic analysis for the Global Burden of Disease Study 2013. Lancet. 2015;386(10010):2287–323.View ArticleGoogle Scholar
- Stakeholder Definition [http://www.lse.co.uk/financeglossary.asp?searchTerm=stakeholder&iArticleID=927&definition=stakeholder]. Accessed 7 Dec 2016.
- Goodyear-Smith F, Jackson C, Greenhalgh T. Co-design and implementation research: challenges and solutions for ethics committees. BMC Med Ethics. 2015;16:78.View ArticlePubMedPubMed CentralGoogle Scholar
- King KF, Kolopack P, Merritt MW, Lavery JV. Community engagement and the human infrastructure of global health research. BMC Med Ethics. 2014;15:84.View ArticlePubMedPubMed CentralGoogle Scholar
- Dowdy DW. Partnership as an ethical model for medical research in developing countries: the example of the “implementation trial”. J Med Ethics. 2006;32(6):357–60.View ArticlePubMedPubMed CentralGoogle Scholar
- George AS, Mehra V, Scott K, Sriram V. Community participation in health systems research: a systematic review assessing the state of research, the nature of interventions involved and the features of engagement with communities. PLoS One. 2015;10(10):e0141091.View ArticlePubMedPubMed CentralGoogle Scholar
- Hofmeyer A, Scott C, Lagendyk L. Researcher-decision-maker partnerships in health services research: practical challenges, guiding principles. BMC Health Serv Res. 2012;12:280.View ArticlePubMedPubMed CentralGoogle Scholar
- Gallo A, Weijer C, White A, Grimshaw JM, Boruch R, Brehaut JC, Donner A, Eccles MP, McRae AD, Saginur R, et al. What is the role and authority of gatekeepers in cluster randomized trials in health research? Trials. 2012;13:116.View ArticlePubMedPubMed CentralGoogle Scholar
- Lange MM. Vulnerability as a concept for health systems research. Am J Bioeth. 2014;14(2):41–3.View ArticlePubMedGoogle Scholar
- Paina L, Peters DH. Understanding pathways for scaling up health services through the lens of complex adaptive systems. Health Policy Plan. 2012;27(5):365–73.View ArticlePubMedGoogle Scholar
- Jennings B, Dawson A. SOLIDARITY in the moral imagination of bioethics. Hast Cent Rep. 2015;45(5):31–8.View ArticleGoogle Scholar
- Lee LM. Public health ethics theory: review and path to convergence. J Law Med Ethics. 2012;40(1):85–98.View ArticlePubMedGoogle Scholar
- Giraudeau B, Caille A, Le Gouge A, Ravaud P. Participant informed consent in cluster randomized trials: review. PLoS One. 2012;7(7):e40436.View ArticlePubMedPubMed CentralGoogle Scholar
- Hutton JL, Eccles MP, Grimshaw JM. Ethical issues in implementation research: a discussion of the problems in achieving informed consent. Implement Sci. 2008;3:52.View ArticlePubMedPubMed CentralGoogle Scholar
- McRae AD, Weijer C, Binik A, Grimshaw JM, Boruch R, Brehaut JC, Donner A, Eccles MP, Saginur R, White A, et al. When is informed consent required in cluster randomized trials in health research? Trials. 2011;12:202.View ArticlePubMedPubMed CentralGoogle Scholar
- Benatar SR. Reflections and recommendations on research ethics in developing countries. Soc Sci Med. 2002;54(7):1131–41.View ArticlePubMedGoogle Scholar
- Council for International Organizations of Medical Sciences [CIOMS]: International Ethical Guidelines for Biomedical Research Involving Human Subjects. In. Edited by Organization CfIOoMSCaWH: Council for International Organizations of Medical Sciences (CIOMS); 2002.Google Scholar
- Maher D. Patient partnerships for research. Public Health Action. 2015;5(1):1.View ArticlePubMedPubMed CentralGoogle Scholar
- Hyder AA, Dawson L. Defining standard of care in the developing world: the intersection of international research ethics and health systems analysis. Dev World Bioeth. 2005;5(2):142–52.View ArticlePubMedGoogle Scholar
- London AJ. The ambiguity and the exigency: clarifying ‘standard of care’ arguments in international research. J Med Philos. 2000;25(4):379–97.View ArticlePubMedGoogle Scholar
- Lie RK, Emanuel E, Grady C, Wendler D. The standard of care debate: the Declaration of Helsinki versus the international consensus opinion. J Med Ethics. 2004;30(2):190–3.View ArticlePubMedPubMed CentralGoogle Scholar
- Krubiner CB, Syed RH, Merritt MW. Health researchers’ ancillary-care responsibilities in low-resource settings: the landscape of institutional guidance. IRB. 2015;37(3):12–9.PubMedGoogle Scholar
- Merritt MW. Health researchers’ ancillary care obligations in low-resource settings: how can we tell what is morally required? Kennedy Inst Ethics J. 2011;21(4):311–47.View ArticlePubMedPubMed CentralGoogle Scholar
- Hyder AA, Merritt MW. Ancillary care for public health research in developing countries. JAMA. 2009;302(4):429–31.View ArticlePubMedPubMed CentralGoogle Scholar
- Pratt B, Hyder AA. Global justice and health systems research in low- and middle-income countries. J Law Med Ethics. 2015;43(1):143–61.View ArticlePubMedGoogle Scholar
- Wyber R, Vaillancourt S, Perry W, Mannava P, Folaranmi T, Celi LA. Big data in global health: improving health in low- and middle-income countries. Bull World Health Organ. 2015;93(3):203–8.View ArticlePubMedPubMed CentralGoogle Scholar
- Hunt MR, Godard B. Beyond procedural ethics: foregrounding questions of justice in global health research ethics training for students. Global Public Health. 2013;8(6):713–24.View ArticlePubMedPubMed CentralGoogle Scholar
- Pratt B, Loff B. A framework to link international clinical research to the promotion of justice in global health. Bioethics. 2014;28(8):387–96.View ArticlePubMedGoogle Scholar
- Lairumbi GM, Parker M, Fitzpatrick R, English MC. Forms of benefit sharing in global health research undertaken in resource poor settings: a qualitative study of stakeholders’ views in Kenya. Philos Ethics Humanit Med. 2012;7:7.View ArticlePubMedPubMed CentralGoogle Scholar
- Schulz-Baldes A, Vayena E, Biller-Andorno N. Sharing benefits in international health research. Research-capacity building as an example of an indirect collective benefit. EMBO Rep. 2007;8(1):8–13.View ArticlePubMedPubMed CentralGoogle Scholar
- Universal Declaration on Bioethics and Human Rights [http://portal.unesco.org/en/ev.php-URL_ID=31058&URL_DO=DO_TOPIC&URL_SECTION=201.html]. Accessed 7 Dec 2016.
- Davison CM, Kahwa E, Atkinson U, Hepburn-Brown C, Aiken J, Dawkins P, Rae T, Edwards N, Roelofs S, MacFarlane D. Ethical challenges and opportunities for nurses in HIV and AIDS community-based participatory research in Jamaica. J Empir Res Hum Res Ethics. 2013;8(1):55–67.View ArticlePubMedGoogle Scholar