Table 2 Types of data and methods of data collection employed in the Learning Evaluation of the Advancing Care Together (ACT) study
Primary data type | Description of data | Data collection process |
|---|---|---|
Documents | Documents collected included: call for proposals, notes from program office-sponsored meetings, and email communications when available; grant applications and grantee reports; as well as manuscripts, training materials, grantee presentations. | • We collected documents throughout the study period. The program office and grantees shared documents with us freely. |
Online diaries | The online diaries produced written documentation of what was observed and experienced during implementation of each intervention. Several hundred pages of rich description of implementation processes was documented across practices using the online diaries. | • We worked with practice leaders to identify people who were closely involved with implementing integration strategies and who could write about their observations/experiences during implementation. |
• Four to six people were identified from each practice | ||
• Each practice had a private diary room. Only the diary keepers and the evaluators had access to the room. | ||
|  |  | • Diary keepers posted entries approximately every 2 weeks. Posted entries were viewed and responded by other diary keepers from the team. |
|  |  | • Evaluators also interacted with diary keepers in real-time, asking questions, and discussing and responding to diary entries. |
|  |  | • Diary data was collected throughout the study period. |
Site visits | Evaluation team members conducted 2-day visits to each practice. Fieldnotes were prepared by the evaluators on the site visit to document observations about the practice and the integration strategies being implemented. | Evaluators conducted interviews with key informants at each practice. Additionally, evaluators observed practice members doing the intervention at practices, when this was possible. |
Interviews | Group interviews were conducted with practice members at program office-sponsored meetings. Fieldnotes were prepared to capture what was said during these interviews. | Interviews were conducted once a year (at baseline, 1 year into the study period, and at the conclusion of the study). When we had unanswered questions about an intervention, we scheduled a phone interview. |
Surveys | A web-based survey was collected from each participating site to assess practice structure and function, including patient panel characteristics (socio-demographic and insurance), practice type and ownership, provider types, use of registries and clinical decision support systems, and existing practices pertaining to delivering integrated care. | Practice survey data was collected at baseline (pre-intervention). One person in the practice (e.g., Office Manager, lead physician) completed this survey. |
EHR data | • Process of care measures (screening for behavioral and/or primary care conditions and receipt or referral for further counseling as needed) for target patients were collected to examine if implementing interventions resulted in improvements in care processes | Process measures were extracted from the EHR by a designated practice member and reported to the evaluation team every 3 months over a 1-year period. |
• Outcome measures were collected for screen-positive patients to examine if interventions resulted in change in outcomes | A designated practice member extracted visit-level data on outcome measures, socio-demographics, and comorbidity data for each patient who screened positive for primary and/or behavioral health condition at baseline and up to 6 months after end of evaluation period. |