Review objective and scope
The primary goal of this paper is to provide a review of existing measure resources relevant to D&I research and to describe their characteristics and possible use by researchers and other practitioners/end-users. Specifically, the target was resources that provided information about D&I measures (i.e., websites or systematic reviews that synthesized information about existing D&I measures), not individual D&I measures as these are captured within the resources we sought to identify. Measure resources include living repositories (e.g., websites and wiki pages) and static resources (e.g., systematic and scoping reviews).
To increase the comprehensiveness and exhaustiveness of the search, we employed a two-step search process that concluded in May 2015. First, we conducted an environmental scan using a respondent-driven, non-probabilistic sampling approach to identify key informants who could help us identify additional resources beyond the peer-reviewed literature, in the gray literature or in the development stages. This approach, which leverages the informational power of social networks, can augment traditional environmental scans and literature reviews in situations in which the searched-for items (i.e., measures resources) are not clearly and consistently indexed with standard terms in bibliographic databases. The following listservs were accessed: the SIRC, the Association for Behavioral and Cognitive Therapy Dissemination and Implementation Science Special Interest Group, and the Implementation Network listserv. We also searched websites and electronic newsletters for additional resources.
Second, a review (scoping and systematic) of the published literature was conducted using two approaches. First, an initial set of publications (reviews of dissemination and/or implementation measures) was identified through recommendations from attendees of the NIH meeting. Then, a systematic review of the literature was completed using two search engines (PubMed, Web of science) to identify papers published between 2000 and 2014 in English language using a set of search term combinations (dissemination/implementation + measure/measurement/instrument/scale/evaluation + review). Titles and abstracts were filtered for reviews (both systematic and non-systematic) of dissemination and/or implementation measures.
Five inclusion/exclusion criteria were set. First, the resources needed to include measures related explicitly to D&I. Therefore, resources that included information about measures used to evaluate D&I outcomes were included, whereas resources focused solely on quality improvement or quality of care measures or patient-level health outcomes were excluded. Second, resources were excluded if the investigative team was unable to access the resource beyond its cited name or if the resource was not yet fully developed. Third, for static resources, we included published reviews (systematic or not) that focused on one or multiple D&I-relevant constructs, which includes D&I outcomes (e.g., adoption, sustainment; ) or factors implicated in the D&I process (e.g., leadership, climate; ). Fourth, only reviews were included from the published literature. Finally, resources that discussed either/both quantitative and qualitative measures were included.
The focus of the data extraction was collaboratively developed to obtain useful summary information that could be gleaned from the resources and would be applicable across resources, to ultimately aid researchers and stakeholders in determining the resource of most relevance. The data extraction resulted in 13 unique pieces of data that reflected both quantitative and qualitative information about the resources including characterizing features of the resource (organizing framework, audience, discipline/scope, type of measure, measure identification approach, resource status, and access) and summary data of the measures information (number of constructs, number of measures, if the measures are included in the resource, measure metadata, psychometric information, pragmatic rating, and analysis level).
Once the resource sample and the data extraction process were finalized, the data extraction was completed by two independent research assistants trained by the first authors (BR and CCL). Research assistants independently extracted data from each resource and then met for consensus in order to achieve one set of summary data for each resource . When consensus could not be achieved, the first authors were consulted to make a final determination.