Implementation of family psychosocial risk assessment in pediatric cancer with the Psychosocial Assessment Tool (PAT): study protocol for a cluster-randomized comparative effectiveness trial
Implementation Science volume 15, Article number: 60 (2020)
Childhood cancer affects and is affected by multiple levels of the social ecology, including social and relational determinants of health (e.g., economic stability, housing, childcare, healthcare access, child and family problems). The 2015 Standards of Psychosocial Care in Pediatric Cancer outline optimal psychosocial care sensitive to these ecological factors, starting with assessment of psychosocial healthcare needs to promote medical and psychosocial outcomes across all children with cancer. To address the first standard of family psychosocial assessment, the Psychosocial Assessment Tool (PAT) is a validated screener ready for broad implementation.
The PAT will be implemented across a national sample of 18 pediatric cancer programs ranging in size (annual new patients) in a mixed methods, comparative effectiveness study, guided by the Interactive Systems Framework for Dissemination and Implementation, comparing two implementation strategies. It is hypothesized that implementation will be more successful at the patient/family, provider, and institutional level when training (strategy I) is combined with implementation expanded resources (strategy II). There are three aims: (1) Refine the two implementation strategies using semi-structured qualitative interviews with 19 stakeholders including parent advocates, providers, pediatric oncology organization representatives, healthcare industry leaders; (2) Compare the two theoretically based and empirically informed strategies to implement the PAT in English and Spanish using a cluster-randomized controlled trial across 18 sites. Stratified by size, sites will be randomized to cohort (3) and strategy (2). Outcomes include adoption and penetration of screening (patient/family), staff job satisfaction/burnout (provider), and cost-effective use of resources consistent with family risk (institution); (3) Based on the results of the trial and feedback from the first and second aim, we will develop and disseminate a web-based PAT Implementation Toolkit.
Use of the PAT across children’s cancer programs nationally can achieve the assessment standard and inform equitable delivery of psychosocial care matched to family need for all patients.
The diagnosis and treatment of pediatric cancer affects and is affected by multiple levels of the social ecology, including patient and caregiver physical and psychosocial health. Particularly at risk are families with limited instrumental (i.e., financial) and social resources and pre-existing child and family problems. Institute of Medicine Reports [1, 2] and the Standards of Psychosocial Care in Pediatric Cancer  call for improvement in delivery of psychosocial care. The standards outline evidence-based care for all patients and families to improve health, increase access to care, and reduce health disparities by decreasing distress, addressing risks, and improving quality of life. The first standard is “youth with cancer and their family members should routinely receive systematic assessment of their psychosocial healthcare needs” . Universal screening at diagnosis fosters early identification of psychosocial risks and provides the opportunity to match psychosocial care to the level of family need for more equitable, effective, and integrated services. Confirming this, in qualitative interviews with multidisciplinary healthcare providers regarding the implementation of screening, the overarching theme was that screening all families is important because it facilitates clinical care and partnerships that can improve outcomes especially for those at risk for disparities. However, few programs offer such care in an efficient, comprehensive, consistent manner [5, 6], highlighting critical gaps in care that can magnify health disparities.
This study addresses this critical gap in the delivery of care to our diverse population of children with cancer and their families by evaluating two approaches to implementing an evidence-based, parent report screener of family psychosocial risk in English and Spanish—the Psychosocial Assessment Tool (PAT) [7, 8]. Risk screening initiates a process of preventive interventions across cancer treatment and facilitates access to evidence-based psychosocial care for children, potentially preventing increased distress and long-term limitations to health-related quality of life [9, 10]. Universal, systematic screening assures that assessments are integrated and resources meet the needs of all children with cancer and their families. However, barriers to universal, systematic screening and linked evidence-based care have been identified. The Preparing to Implement the Psychosocial Standards–Current Staffing and Services (PIPS-CSS) study of 144 US pediatric cancer programs conducted to prepare for broad implementation of the standards [5, 11] found that there are challenges and inconsistent interpretations of psychosocial care [5, 11]. Similarly, data from the Children’s Oncology Group (COG)  and a national survey of social workers  demonstrated inconsistent and often inadequate services. Barriers to implementation are evident, and providers note the importance of training in terms of how screening is accomplished .
The Psychosocial Assessment Tool (PAT)
The Psychosocial Assessment Tool (PAT) [7, 8] is an evidence-based parent/caregiver report screener of family psychosocial risk in English and Spanish. The PAT generates a total score and 7 subscales (family structure, social support, child problems, sibling problems, family problems, stress reactions, family beliefs). Since the initial versions of the PAT [14,15,16], we have refined the PAT with the current all literacy version reflecting the broad assessment of family psychosocial risks. Screening with the PAT can be completed at diagnosis , can be used by multidisciplinary staff, and facilitates the delivery of psychosocial care . Embedded in social ecology theory, the PAT total score maps on to the Psychosocial Preventative Health Model (PPPHM, Fig. 1) , a three-tier model which represents the distribution of psychosocial risks across the population of families. Most families experience some distress but have minimal risk factors (low levels of distress, few prior child, or family problems) and resources (financial resources, strong social support) that help them cope and adapt to their child’s illness (universal). A smaller group of families (targeted) have identified areas of risk and moderate resources. At the top of the pyramid are families with more severe problems, many risk factors, and few resources (clinical).
Empirical evidence supports the readiness of the PAT for broad implementation. It is used in 28.9% of pediatric cancer programs in the USA  and widely in other countries [20,21,22,23,24,25,26,27,28]. The PAT is acceptable to families across race, ethnicity, and SES [7, 8, 10, 16, 17, 29] and has been shown to impact psychosocial outcomes at higher levels of distress when results are shared with staff . We adapted the PAT for hematopoietic stem cell transplantation (HCST)  including development of a clinical pathway to facilitate its integration in clinical care [32, 33], and in sickle cell disease, adding items to capture relevant aspects of the social context for families (e.g., school absences, changes in housing) .
We do not know the extent to which the PAT has been adopted, whether implementation is consistent across families, and if PAT implementation is sustained. The best practices for implementation have not been studied, and barriers and facilitators to systematic implementation have not been evaluated. Challenges such as time, determining who will screen, technical difficulties, and linking screening to care were potential barriers. In a pilot of an implementation model using a workshop and consultation calls in three states, 9 of 12 centers successfully implemented the web-based PAT, half using both the English and Spanish versions . The pilot data informs the implementation and measurement strategies in this study.
Based on the Interactive Systems Framework for Dissemination and Implementation (ISF) , there are three stages in this mixed methods research (Fig. 2). First, two implementation strategies , to improve integration of the PAT into standard pediatric cancer care, will be refined using feedback from 19 stakeholders (qualitative methods). The strategies are based on prior PAT studies, the dissemination and implementation literature , and Social Ecological  and Pediatric Psychosocial Preventative Health Models . Strategy I is training (webinar) to educate providers on the PAT and its administration. Strategy II is Training + Implementation Expanded Resources (TIER), which augments training with consultation calls and identification of a site champion. Second, we will conduct a comparative effectiveness trial of the two strategies at 18 childhood cancer centers of three sizes examining family (penetration, health equity), provider (feasibility, acceptability, burnout, and job satisfaction), and institution (adoption, sustainability, costs) implementation outcomes . We will randomize sites to time of implementation (three cohorts) and strategy (two—I, II). Third, we will develop and disseminate a web-based PAT Implementation Toolkit for family psychosocial risk screening in pediatric cancer.
Refine strategies I (training) and II (TIER) using semi-structured interviews with stakeholders—parent advocates, multidisciplinary health care providers, national pediatric oncology professional organizations, and health care industry leaders (implementation team).
Compare the two theoretically based and empirically informed strategies to implement the PAT in English and Spanish using a cluster-randomized controlled trial. Compared to training:
H2.1. At the patient/family level, TIER will be associated with: (a) a higher proportion of families of newly diagnosed children screened and provided with feedback (penetration) and (b) higher rates of screening for ethnic minority and socioeconomically diverse families (health equity).
H2.2. At the provider level, TIER will be (a) more feasible and rated as appropriate and acceptable, (b) associated with greater engagement in addressing health disparities, and (c) associated with less burnout and better job satisfaction.
H2.3. At the institution level, TIER will be associated with: (a) a higher rate of site participation (adoption), (b) more positive perceptions of implementation benefits and fewer challenges (sustainability), and (c) psychosocial care better matched to need demonstrating a more equitable distribution of services and costs of care.
Based on the results of the trial and further guidance from the implementation team, we will integrate acceptable, feasible, and effective strategies to develop and disseminate a web-based PAT Implementation Toolkit.
Methods and design
Overview of the study
The aim of this mixed methods research is to implement universal, systematic family psychosocial risk screening with the PAT in English and Spanish to assure that all families of children newly diagnosed with cancer at the participating cancer centers are screened. The setting/context of the research is pediatric cancer programs in the USA. The approach, reflected in the three aims, is guided by the ISF . We selected specific implementation strategies from the Expert Recommendations for Implementing Change (ERIC) project  targeting implementation outcomes at three levels (patient/family, provider, institution). ERIC strategies utilized are noted in parentheses throughout this article.
Aim 1 corresponds to the first component of the ISF, Prevention Synthesis and Translation System. We will prepare programs to implement the PAT by conducting qualitative semi-structured interviews with a diverse set of stakeholders (n = 19). Interviews will be focused on details of strategy I (training via webinar) and strategy II (training + TIER—consultation calls and identification of a champion), followed by questions about facilitators and barriers, and about implementation strategies and resources needed for universal screening and care delivery to address health inequities. We anticipate fine-tuning and adding components to improve penetration and health equity targets, acceptability and feasibility, adoption, and sustainability.
The activities of aim 2 correspond to the second component of the ISF, Prevention Support System. Support for those implementing innovation occurs at multiple levels within the system—patients/families, providers, and institution. To implement the PAT in English and Spanish, 18 pediatric cancer programs, of varying sizes and with geographic distribution assuring representation of ethnic and racial minority families and families at socioeconomic risk, have agreed to participate. We will conduct a cluster-randomized comparative effectiveness trial of the two implementation strategies across three cohorts stratified by size of site based on new patients per year (Fig. 3).
Aim 3 activities correspond to the third component of the ISF, Prevention Delivery System, in the development and dissemination of the PAT Implementation Toolkit.
Stakeholder interviews to refine implementation strategies (aim 1)
Qualitative interviews will be conducted with a national group of diverse stakeholders, selected using purposive criterion-based sampling  to represent different levels of the social ecology. The data from the interviews will inform components of the two implementation strategies and incorporate questions related to broader implementation (e.g., facilitators and barriers). The stakeholder interviews focus on ERIC implementation strategies (use advisory boards and workgroups, prepare patients/consumers to be active participants, and build a coalition).
Participants (n = 19) are parent advocates, multidisciplinary healthcare providers, members/leaders of key pediatric oncology professional organizations, and leaders in the pediatric healthcare industry.
Each stakeholder will be interviewed by video conference using a theoretically driven and empirically based semi-structured interview guide . The purpose of the interview is to refine strategies I and II, identify additional barriers or facilitators of implementation across the patient/family, provider, and institution levels, and ensure PAT implementation strategies address these barriers to reduce disparities in care.
Cluster-randomized comparative effectiveness trial (aim 2)
PAT implementation will be tested using a head-to-head randomized implementation (comparative effectiveness of two implementation strategies) trial initiated in three cohorts of 1 year each (2021, 2022, 2023). All 18 pediatric cancer programs invited to participate in the trial agreed to a 2-step randomization process stratified by size of site; sites will be randomized to one of three cohorts and then to strategy, comparing strategy I (training) with strategy II (TIER).
Selection of sites
The following criteria were used in site selection:
Provide staff and tablet computer for screening. Each site agreed to provide the staff person(s) screening and tablets. Each site will be provided support for a portion of the site PI’s effort and a part-time research coordinator based on center size for purposes of research only (the research coordinator will not do the screening).
Center size. Because program size is related to the size of the psychosocial team and related psychosocial resources , we used PIPS-CSS data to stratify by size to obtain three equal and clinically relevant groups based on number of new pediatric cancer patients annually—small (30–60), medium (61–149), and large (150+). We selected 6 sites for each of the three size categories.
Psychosocial staff. To assure that there are staff to conduct screening and act on the results of screening, selected sites are at or above the median for the size of their psychosocial team (number [full time equivalents] social workers + psychologists + psychiatrists + child life specialists)  based on PIPS-CSS data.
Diversity of population. We selected centers in states with a high percentage of ethnic or racial minority families and/or families with socioeconomic disadvantage. CA, TX, FL, NY, and NJ have large Hispanic populations . AL, NC, and VA have the most rapidly increasing Hispanic populations, and AL, LA, NC, and VA have the largest percentage of African-Americans . Thirteen out of 18 sites are in these states. Twelve out of 18 are in states with > 20% of children living in poverty . Other sites were selected to balance geography, race, ethnicity, and SES.
Since screening will be integrated into the clinical services of the centers, it will be routine clinical care. Coded EHR data with minimal protected health information (PHI) will be collected for patients. Reflecting the importance of staff in implementation , consented participants include the site PI and screeners and, in strategy II, the champion.
All sites, regardless of randomization to strategy I or II, will participate in a 3-h professionally prepared webinar (develop educational materials, distribute educational materials) at the beginning of their cohort year. The webinar will include all information necessary to understand, access, and deliver the web-based PAT. The webinar will be based on our in-person training program and curriculum  modified to integrate feedback from aim 1 interviews. Each site PI, all screeners, and the research coordinator will participate. For sites randomized to strategy II, the champion will attend. We will work with sites to identify this team and the champion (in TIER) “individuals who dedicate themselves to supporting, marketing, and driving through an implementation, overcoming indifference or resistance that the intervention may encounter in an organization” .
After the PAT is completed online, it is scored immediately, and a summary of the score and clinical concerns identified is generated. Only coded data will be transmitted to the study data core. A master list for this data will be maintained at each respective study site and will not be shared with the core research team. We will provide support to sites in the technical aspects of the implementation related to using the web-based forms (centralize technical assistance). If we identify any pattern of problems with technical aspects of implementation, we will communicate with sites promptly.
PAT implementation plan
Each site will complete an implementation plan, describing who will be screened, where results will be stored, how results will be communicated to families and to staff, and how results will be used (develop a formal implementation blueprint). For TIER, the plan includes the responsibilities of the champion.
For sites randomized to strategy I, the webinar is the implementation condition. Sites will receive technology support, as needed, throughout the implementation period. These strategies correspond to implementation strategies—creating a structure for implementation including creating implementation teams and developing an implementation plan .
Strategy II includes the webinar and technical support as above with the addition of two evidence-based resources that may improve implementation. The site PI and center staff conducting screening will participate in a monthly 1-h consultation call (provide ongoing consultation, create a learning collaborative) with other TIER sites in that cohort. The group format of this strategy is intended to foster group problem-solving and peer support about issues in implementation. Sites will identify a champion (identify and prepare champions) who will advocate for PAT implementation and support staff in screening activities by serving as a resource to problem-solve and communicate with the broader clinical staff about screening and psychosocial risk. The champion will likely be a clinical leader with enthusiasm and commitment to universal psychosocial risk screening.
Measurement/outcomes (all sites)
The measures assess outcomes across patient/family, provider, and institution levels (Table 1) that are clearly operationalized and reproducible . Site PIs and coordinators will attend training on data collection procedures, in separate sessions for the two conditions.
At the patient/family level (Hyp 2.1) the site coordinator will extract EHR data and send the coded data via REDCap to the data core. The following data for English and Spanish versions of the PAT will be reported monthly: new patients meeting eligibility requirements per the PAT implementation plan; patients with documentation of PAT screening; patients receiving feedback letter; demographic data on all eligible and all screened (race, ethnicity, zip code, insurance).
At the provider level (Hyp 2.2), the site PI and screeners will complete the following self-report measures (~ 15 min) at baseline (T1), and 6 (T2) and 12 (T3) months via REDCap: acceptability of intervention measure; intervention appropriateness measure and feasibility of intervention measure ; measure of physician engagement in addressing racial and ethnic health care disparities , satisfaction of employees in healthcare survey , and Maslach Burnout Inventory .
The outcomes at the institution level (Hyp 2.3) are as follows: adoption, the intention of sites to use the PAT (the ratio of sites that initiate the study); whether screening is perceived as an asset to the institution using the PAT Implementation Questionnaire (PIQ)  which will be completed by the site PI, the screeners, and the champion at T1, T2, and T3; whether services are matched to the needs of families based on PPPHM levels using EHR data using the Psychosocial Services and Medical Treatment Checklist (PSMTC)  at 30, 60, and 90 days; cost using the data from the PSMTC to estimate the amount of psychosocial staff time and national median hourly salary data [51,52,53]. The Intensity of Treatment Rating Scale (ITR-3)  will be completed at the 90-day post-screening data collection time point based on diagnosis, stage, and treatment.
Whether the PAT was administered as intended will be assessed using monthly EHR data. The coded data captured at Nemours will also allow us to confirm the completion of each PAT in its entirety. The coordinators will meet monthly to problem-solve any concerns about data collection. In the TIER condition, attendance on calls and minutes of the monthly consultation calls will be prepared. The champion will complete a questionnaire at the beginning of the study year, and at 6 and 12 months to document their activities.
Dissemination toolkit (aim 3)
To broadly disseminate the PAT, a theoretically and data-driven web-based PAT Implementation Toolkit will be developed with the information, resources, and tools necessary to implement the PAT. This corresponds with the Prevention Delivery System of the ISF (develop educational materials, distribute educational materials, purposely re-examine the implementation) .
There will be 20 purposively sampled participants from the aim 1 stakeholders (n = 10) and from sites that implemented the PAT in aim 2 (n = 10 site PIs and champions). Cognitive interviews will be conducted to ascertain their interpretation and understanding of the PAT Implementation Toolkit. This sample size is consistent with the PROMIS methods  and the past experience of the research team .
Subsequent to the final analysis of data from the aim 2 trial, we will refine the preliminary framework for the PAT Implementation Toolkit website which we expect will include the training webinar, technology support, PAT Implementation Plan, complete information to implement the PAT across centers of different sizes, materials specific to using the Spanish version of the PAT, suggestions for overcoming identified barriers and bolstering facilitators, evidence for how screening can impact health disparities, frequently asked questions, and guide for sites in identifying resources to guide intervention based on PAT scores. The Toolkit will be web-based and easily accessible.
Qualitative, think-aloud interviews  will be conducted by video conference using a semi-structured interview guide. During this audio-recorded interview, providers “walk through” the web-based PAT Implementation Toolkit to identify areas for clarification and improvement. Interviews will be transcribed by a professional transcription service and uploaded to a qualitative data management program (Atlas.ti ©) for formal coding.
The Agency for Healthcare Research and Quality Publishing and Communications Guidelines (https://www.ahrq.gov/research/publications/pubcomguide/index.html) will guide dissemination. We will coordinate with Children’s Oncology Group (COG) to assure sites have access to the Toolkit. We will contact all sites in the PIPS-CSS study to support their use of the Toolkit. As a site in the National Cancer Institute Community Oncology Research Program (NCORP), we will include a broad network of community and minority sites.
The interviews will be electronically recorded and professionally transcribed. All data will be stored and managed on a secure drive. Data will be inductively analyzed using Atlas.ti ©. Content analytic strategies will be used to identify codes, categories, and themes [58, 59]. Analysis will proceed as the investigators simultaneously collect information through interviews, read each interview as an individual case, consider the topics addressed in the interview guide, disassemble each interview through coding with preliminary codes, define and combine codes into categories and categories into themes, and consider data from each category across all cases .
We will first independently code three interviews selected to represent different stakeholders (patient/family, providers, and institution) to define an initial set of codes and codebook. Study coordinators will be trained to manage and analyze qualitative data [61,62,63] and supervised by an expert qualitative researcher. The investigators will independently code five interviews. For the remaining interviews, each transcript will be coded independently by a two-person team that will discuss each interview and reconcile any discrepancies until they reach 75% agreement. After this point, interviews will be coded independently but will be reviewed and discussed if a discrepancy is found. The investigators will review all the coded transcripts. As coding continues, codes will be combined into categories. Finally, all data will be examined in each category to combine categories into themes. The themes will be translated into content of the PAT webinar for strategy I and enhanced strategies included in strategy II with attention to health disparities and barriers and facilitators at the patient/family, provider, and institution levels. The rigor of the iterative analytic process will follow standards for qualitative research [63, 64].
PAT implementation will be tested using a head-to-head randomized implementation (comparative effectiveness of two implementation strategies) trial in three cohorts over 3 years. Randomization. Eighteen sites will be randomized to one of three cohorts. Each cohort will be further randomized to one of the two strategy conditions. Randomization will stratify sites by size to maximize internal validity and statistical power. Sites will be randomized by a data analyst not connected with the study using a two-step Excel = RAND() function. The sequence will be stored in a password-protected electronic file. Sample size. The estimated sample of patients/families is based on the median number of new patients each year from the PIPS-CSS database: small (n = 60), medium (n = 92), large (n = 339) multiplied by 6 sites at each size, a total of 2946, with 1473 families allocated to each strategy. With an anticipated Interclass Correlation Coefficient (ICC)  of .005, the effective sample size is 950 per arm, a sample size that is sufficient to detect a small effect between the two strategies given 80% power (alpha = .05, one sided). We project a Hispanic sample of 20% of the total (n = 589). Based on our research , approximately 30% of Hispanic caregivers will be more acculturated/not literate in Spanish at the level necessary to complete the Spanish PAT, rendering a sample of 412. With an anticipated ICC of .005, the effective sample size is 224 which is sufficient to detect a medium effective between the English and Spanish versions given 80% power (alpha = .05, one sided). At the provider level, each of 18 sites has a PI and up to four people screening, and for TIER sites, a champion. Therefore the staff sample ranges from a minimum of 45 ([18 × 2] + 9) to a maximum of 99 ([18 × 5] + 9), likely in the middle given the range of size of sites. At the institution level, 18 sites will be randomly assigned to one of two strategies (9 sites/group). These sample sizes are comparable to other implementation science studies .
Data cleaning and missing data
All data will be reviewed for valid values/data entry errors, outliers, and extent/pattern of missing data. Descriptive statistics will be reviewed. Consistency and logic checks will be applied for review/cleaning. The multiple group analysis models will provide valid estimates of efficacy if the proportion of missing values is < 10%. Analysis will be conducted at the patient/family, provider, and institution levels. The effect of program size and cohort will be examined and controlled if needed.
H2.1. ANOVA will compare the effectiveness of the two implementation strategies on penetration and health equity. The outcomes are proportions: families screened/families eligible, families provided feedback/families screened, ethnic minority families screened/ethnic minority families eligible, low SES families screened/low SES families eligible. ICCs among the clusters will be calculated and used to adjust for the cluster effect [67, 68].
H2.2. Three sets of outcome variables—perception of implementation, engagement in addressing health disparities, and burnout/job satisfaction—will be tested. A two group analysis using Structural Equation Modeling (SEM) will compare the effectiveness of the two strategies [69, 70]. To test the effect of time, we will conduct latent growth curve analysis [71,72,73]. Analyses will be conducted using Mplus 5.0  with ML estimation for outcome variables that meet the distribution assumptions, and WLSMV estimation for variables that do not. Potential mediating effects of favorable perception of implementation on provider job satisfaction and burnout will be examined using mediation models. TIER is expected to be associated with more favorable perceptions of implementation, less burnout, and higher job satisfaction.
H2.3. At the institution level, adoption of the PAT will be measured by a ratio of sites that initiate implementation/sites that agreed (H2.3a). If substitutions are necessary, sites that are newly invited will be added to the denominator and adoption calculated by total acceptances/total invited. ANOVA will be conducted to compare the effectiveness of the two strategies. For H2.3b sustainability (PIQ perceptions of implementation benefits and challenges), ANOVA will be conducted to compare the effectiveness of the two strategies on benefits and challenges. For H2.3c, we are interested in the extent to which psychosocial care is matched to need. A cost-effectiveness threshold or criterion to which to compare costs of screening with these two implementation strategies has not been established, necessitating our consideration of valued outcomes in the psychosocial screening literature and resources available. We will use a data analytic approach that we used previously . ANOVA will be conducted to test whether psychosocial care is matched with levels of psychosocial risks, resulting in a 2 (strategy) × 3 (psychosocial risks: clinical, targeted, universal) design on equitable distribution of services and costs of care. It is expected that the 3 PPPHM levels will be related to number and costs of services provided as measured on the PSMTC, with least at universal and most at clinical. It is expected that TIER will result in a better match between level of risk and services provided. Scores from the PSMTC will be derived and mapped onto the levels of the PPPHM. Additional analyses will be conducted to compare English speaking and Spanish speaking families, different ethnicity, race, and SES and insurance status on the outcome variables for H2.3c.
We will summarize the data by item and then aggregate the results across participants to reflect potential problems with Toolkit components and to identify components that are clear and supportive of effective implementation . To ensure rigor, we will systematically analyze and then summarize the interview data following a formal coding scheme. Subsequently, we will develop a cognitive interviewing outcome report, a description of the number and type of participants and interviews completed, a description of the specific procedures used in the interviews and the interview guide, and a written summary of feedback on each of the components . This report will be distributed to the study team who will discuss the issues identified. Decisions will be documented in a tracking matrix . Based on this feedback, with aim 2 data and theoretical frameworks, we will revise and finalize the Toolkit.
Family psychosocial screening, a Standard of Care, if implemented consistently and across children’s cancer programs can reduce disparities in care by facilitating care matched to need, and promote adaptation. The aim of this research is to implement universal, systematic family psychosocial risk screening with the PAT in English and Spanish to assure that all families of children newly diagnosed with cancer at the participating cancer centers are screened. Guided by the ISF for Dissemination and Implementation , this study will result in the development and broad dissemination of a PAT Implementation Toolkit for successful and sustainable implementation of universal, comprehensive, evidence-based family psychosocial screening for all families in pediatric oncology. As one of the few applications of implementation science in pediatric cancer, the results of this trial will provide valuable information about what strategies are effective in supporting comprehensive care. The broad theory proposed by the ISF is innovative in this field and provides a broad, forward looking approach to the process of advancing integrated care.
This project is ambitious and has some potential challenges. If a site that agreed to participate is unavailable, we have other eligible sites at each size to approach. We will retain the balance of sites from states with health disparity populations if we make substitutions. Second, to focus on population-based implementation, we did not consider sites with fewer than 30 new patients annually, and we selected sites with psychosocial staff at the median or above for their size. In addition, our selection criteria (minority population, psychosocial staff) precluded some sections of the USA, particularly less populated states with the smallest centers. Thus, we will not be able to generalize implementation to the smallest sites with fewer resources. However, we will distribute the PAT Implementation Toolkit to these sites and seek to evaluate the impact of broader dissemination of the Toolkit across sites of different sizes and different geographic locations. Finally, for cost-effectiveness, measuring the cost of psychosocial care is complicated and largely without precedent. Involvement of healthcare leaders in aims 1 and 3 will provide expert input in larger system level change to support family psychosocial risk screening and psychosocial care in pediatric cancer.
The importance of evidence-based psychosocial care for children with cancer and their families is recognized. However, too frequently, families of children with cancer do not receive this care, magnifying health disparities in our increasingly diverse pediatric oncology population. These inequities relate directly to screening for family psychosocial risks associated with social and relational determinants of health. Implementation of an evidence-based, parent report screener of family psychosocial risk across the social ecology in English and Spanish may address these health disparities. Risk screening initiates a process of preventive interventions across treatment. Universal, systematic screening assures that assessments are integrated and resources meet the needs of all children with cancer and their families.
Availability of data and materials
The datasets used during this study will be made available by the investigators on reasonable request.
Acceptability of Intervention Measure
Analysis of variance
Awareness, reflection/empowerment, action
Children’s Oncology Group
Electronic health record
Expert Recommendations for Implementing Change
Interclass correlation coefficient
Institutional Review Board
Interactive Systems Framework
Intensity of Treatment Rating Scale
National Cancer Institute Community Oncology Research Program
Psychosocial Assessment Tool
Protected health information
Preparing to Implement the Psychosocial Standards–Current Staffing & Services
PAT Implementation Questionnaire
Pediatric Psychosocial Preventative Health Model
Psychosocial Services and Medical Treatment Checklist
Research Electronic Data Capture
Structural equation modeling
Standards for Reporting Implementation Studies
Training + Implementation Expanded Resources
Robust Weighted Least Squares
Adler NE, Page AEK, editors. Cancer care for the whole patient: meeting psychosocial health needs. Institute of Medicine. 2008; doi: https://doi.org/10.17226/11993.
National Cancer Policy Forum, Board on Health Care Services, Institute of Medicine, National Academies of Sciences, Engineering and Medicine. Comprehensive cancer care for children and their families: summary of a joint workshop by the Institute of Medicine and the American Cancer Society. The National Academies Press. 2015; doi: https://doi.org/10.17226/21754.
Wiener L, Kazak AE, Noll RB, Patenaude AF, Kupst MJ. Standards for psychosocial care for children with cancer and their families: an introduction to the special issue. Pediatr Blood Cancer. 2015;62-S5:S419-S424.
Kazak AE, Abrams AN, Banks J, Christofferson J, DiDonato S, Grootenhuis MA, et al. Psychosocial assessment as a standard of care in pediatric cancer. Pediatr Blood Cancer. 2015;62(Suppl 5):S426–59.
Scialla MA, Canter KS, Chen FF, Kolb EA, Sandler E, Wiener L, et al. Implementing the psychosocial standards in pediatric cancer: current staffing and services available. Pediatr Blood Cancer. 2017. https://doi.org/10.1002/pbc.26634.
Selove R, Kroll T, Coppes M, Cheng Y. Psychosocial services in the first 30 days after diagnosis: results of a web-based survey of Children’s Oncology Gorup (COG) member institutions. Pediatr Blood Cancer. 2012;58(3):435–40.
Kazak AE, Hwang WT, Chen FF, Askins MA, Carlson O, Argueta-Ortiz F, et al. Screening for family psychosocial risk in pediatric cancer: validation of the Psychosocial Assessment Tool (PAT) Version 3. J Pediatr Psychol. 2018;43(7):737–48.
Kazak AE, Hwang WT, Chen FF, Askins MA, Carlson O, Argueta-Ortiz F, et al. Validation of the Spanish version of the Psychosocial Assessment Tool (PAT) in pediatric cancer. J Pediatr Psychol. 2018;43(10):1104–13.
Kazak AE, Schneider S, DiDonato S, Pai AL. Family psychosocial risk screening guided by the Pediatric Preventative Psychosocial Health Model (PPPHM) using the Psychosocial Assessment Tool (PAT). Acta Oncol. 2015;54(5):574–80.
Pierce L, Hocking MC, Schwartz LA, Alderfer MA, Kazak AE, Barakat LP. Caregiver distress and patient health-related quality of life: psychosocial screening during pediatric cancer treatment. Psycho-oncology. 2017;26(10):1555–61.
Scialla MA, Canter KS, Chen FF, Kolb EA, Sandler E, Wiener L, et al. Delivery of care consistent with the Psychosocial Standards in Pediatric Cancer: current practices in the United States. Pediatr Blood Cancer. 2018. https://doi.org/10.1002/pbc.26869.
Jones B, Currin-Mcculloch J, Pelletier W, Sardi-Brown V, Brown P, Wiener L. Psychsocial standards of care for children with cancer and their families: a national survey of pediatric oncology social workers. Soc Work Health Care. 2018;57(4):221–49.
Kazak AE, Barakat LP, Askins MA, McCafferty M, Lattomus A, Ruppe N, et al. Provider perspectives on the implementation of psychosocial risk screening in pediatric cancer. J Pediatr Psychol. 2017;42(6):700–10.
Kazak AE, Prusak A, McSherry M, Simm S, Beele D, Rourke M, et al. The Psychosocial Assessment Tool (PAT)©: pilot data on a brief screening instrument for identifying high risk families in pediatric oncology. Fam. Syst. Health. 2001;19(3):303–17. https://doi.org/10.1037/h0089454.
Kazak AE, Cant MC, Jensen MM, McSherry M, Rourke MT, Hwang WT, et al. Identifying psychosocial risk indicative of subsequent resource utilization in families of newly diagnosed pediatric oncology patients. J Clin Oncol. 2003;21(17):3220–5.
Pai AL, Patino-Fernandez AM, McSherry M, Beele D, Alderfer MA, Reilly AT, et al. The Psychosocial Assessment Tool (PAT2.0): psychometric properties of a screener for psychosocial distress in families of children newly-diagnosed with cancer. J Pediatr Psychol. 2008;33(1):50–62.
Kazak AE, Barakat LP, Ditaranto S, Biros D, Hwang WT, Beele D, et al. Screening for psychosocial risk at cancer diagnosis: the Psychosocial Assessment Tool. J Pediatr Hematol Oncol. 2011;33(4):289–94.
Kazak AE, Barakat LP, Hwang WT, Ditaranto S, Biros D, Beele D, et al. Association of psychosocial risk screening in pediatric cancer with psychosocial services provided. Psycho-oncology. 2011;20(7):715–23. https://doi.org/10.1002/pon.1972.
Kazak AE. Pediatric Psychosocial Preventative Health Model (PPPHM): research, practice and collaboration in pediatric family systems medicine. Fam. Syst. Health. 2006;24(4):381–95.
Sint Nicolaas SM, Schepers SA, van den Bergh EMM, de Boer Y, Streng I, van Dijk-Lokkart EM, et al. Match of psychosocial risk and psychosocial care in families of a child with cancer. Pediatr Blood Cancer. 2017. https://doi.org/10.1002/pbc.26687.
Barrera M, Hancock K, Rokeach A, Atenafu E, Cataudella D, Punnett A, et al. Does the use of the revised Psychosocial Assessment Tool (PATrev) result in improved quality of life and reduced psychosocial risk in Canadian families with a child newly diagnosed with cancer? Psycho-oncology. 2014;23(2):165–72.
Gilleland J, Reed-Knight B, Brand S, Griffin A, Wasilewski-Masker K, Meacham L, et al. Assessment of family psychosocial functioning in survivors of pediatric cancer using the PAT2.0. Psycho-oncology. 2013; 22(9):2133-2139.
Karlson CW, Smith ML, Haynes S, Faith MA, Pierce J, Elkin TD, et al. Risk for psychosocial problems in pediatric cancer: impact of socioeconomics. Child Health Care. 2013;42(3):231–47. https://doi.org/10.1080/02739615.2013.816602.
McCarthy MC, Clarke NE, Vance A, Ashley DM, Heath JA, Anderson VA. Measuring psychosocial risk in families caring for a child with cancer: the Psychosocial Assessment Tool (PAT2.0). Pediatr Blood Cancer. 2009;53(1):78–83.
McCarthy MC, DeGraves S, Wakefield CE, Bowden MJ, Marks LV, Williams LK. The association of psychosocial screening and service provision in pediatric oncology: the Psychosocial Assessment Tool (PAT2.0) into clinical practice. Support Care Cancer. 2016;24(7):2945–52.
McCarthy MC, Hearps SJ, Muscara F, Anderson VA, Burke K, Hearps SJ, et al. Family psychosocial risk screening in infants and older children in the acute pediatric hospital setting using the Psychosocial Assessment Tool. J Pediatr Psychol. 2016;41(7):820–9.
Schepers SA, Sint Nicolaas SM, Maurice-Stam H, Haverman L, Verhaak CM, Grootenhuis MA. Parental distress 6 months after a pediatric cancer diagnosis in relation to family psychosocial risk at diagnosis. Cancer. 2018;124(2):381–90.
Sharkey CM, Schepers SA, Drake S, Pai AL, Mullins LL, Grootenhuis MA. Psychosocial risk profiles among American and Dutch families affected by pediatric cancer. J Pediatr Psychol. 2020;45(4):463–73.
Alderfer MA, Mougianis I, Barakat LP, Beele D, DiTaranto S, Hwang WT, et al. Family psychosocial risk, distress, and service utilization in pediatric cancer: predictive validity of the Psychosocial Assessment Tool. Cancer. 2009;115(Suppl 18):4339–49.
Barrera M, Alexander S, Atenafu EG, Chung J, Hancock K, Solomon A, et al. Psychosocial screening and mental health in pediatric cancer: a randomized controlled trial. Health Psychol. 2020;39(5):381–90.
Pai ALH, Madan-Swain AM, Chen FF, Hwang WT, Vega G, Carlson O, et al. Screening for family psychosocial risk in pediatric hematopoietic stem cell transplantation with the Psychosocial Assessment Tool. Biol Blood Marrow Transplant. 2019;25(7):1374–81.
Kazak AE, Madan-Swain AM, Pai AL, Canter, K, Carlson O, Vega G et al. Caregiver perspectives on psychosocial care in pediatric hematopoietic stem cell transplantation (HCT). Clin Pract Pediatr Psychol. 2020;8(1):67-78; doi:10.1037%2Fcpp0000278.
Kazak AE, Madan-Swain AM, Canter K, Vega G, Joffe N, Deatrick JA, et al. A psychosocial clinical care pathway for pediatric hematopoietic stem cell transplantation. Pediatr Blood Cancer. 2019;66(10); doi:https://doi.org/10.1002/pbc.27889.
Reader SK, Keeler CN, Chen FF, Ruppe NM, Rash-Ellis DL, Wadman JR, et al. Psychosocial screening in sickle cell disease: validation of the Psychosocial Assessment Tool. J Pediatr Psychol. 2020;45(4):423–33.
Kazak AE, Christofferson J, Gutierrez Richards H, Rivero-Conil S, Sandler E. Implementing the Psychosocial Assessment Tool (PAT) in clinical oncology practice. Clin Pract Pediatr Psychol. 2019;7(2):140–50. https://doi.org/10.1037/cpp0000246.
Wandersman A, Duffy J, Flaspohler P, Noonan R, Lubell K, Stillman L, et al. Bridging the gap between prevention research and practice: the Interactive Systems Framework for Dissemination and Implementation. Am J Community Psychol. 2008;41(3-4):171–81.
Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10:21. https://doi.org/10.1186/s13012-015-0209-1.
Kazak AE. Families of chronically ill children: a systems and social-ecological model of adaptation and challenge. J Consult Clin Psychol. 1989;57(1):25–30. https://doi.org/10.1037/0022-006X.57.1.25.
Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65–76. https://doi.org/10.1007/s10488-010-0319-7.
Patton MQ. Purposeful sampling. In: Mathison S, editor. Encyclopedia of evaluation. Thousand Oaks, CA: Sage; 2005. p. 343–4. https://doi.org/10.4135/9781412950558.n453.
Bredart A, Marrel A, Abetz-Webb L, Lasch K, Acquadro C. Interviewing to develop Patient Reported Oucome (PRO) measures for clinical research: elicit patients’ experience. Health Qual Life Outcomes. 2014;12:15.
Stepler R, Lopez MH. Ranking the Latino population in the states. 2016. http://www.pewhispanic.org/2016/09/08/4-ranking-the-latino-population-in-the-states/. .
Karuga J. 10 states with the largest African-American populations. WorldAtlas. 2019; https://www.worldatlas.com/articles/us-states-with-the-largest-relative-african-american-populations.html. .
Child Poverty. NCCP. (n.d.). http://www.nccp.org/topics/childpoverty.html. .
Geerligs L, Rankin NM, Shepherd HL, Butow P. Hospital-based interventions: a systematic review of staff-reported barriers and facilitators to implementation processes. Implement Sci. 2018;13(36); doi:https://doi.org/10.1186/s13012-018-0726-9.
Lewis CC, Fischer S, Weiner BJ, Stanick C, Kim M, Martinez RG. Outcomes for implementation science: an enhanced systematic review of instruments using evidence-based rating criteria. Implement Sci. 2015;10(155); doi:https://doi.org/10.1186/s13012-015-0342-x.
Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, et al. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci. 2017;12(1):108.
Alexander GC, Lin S, Sayla MA, Wynia MK. Development of a measure of physician engagement in addressing racial and ethnic health care disparities. Health Serv Res. 2008;43(2):773–84.
Chang E, Cohen J, Koethe B, Smith K, Bir A. Measuring job satisfaction among healthcare staff in the United States: a confirmatory factor analysis of the Satisfaction of Employees in Healthcare (SEHC) survey. Int J Qual Health Care. 2017;29(2):262–8.
Maslach C, Jackson SE, Leiter MP. Maslach Burnout Inventory Manual 4th Edition. Mind Garden, Inc: Menlo Park, CA; 2016.
Kichler JC, Valenzuela J, Barker D, Noser AE, Brosig CL, Hilliard ME, et al. Society of Pediatric Psychology workforce survey: updated factors related to compensation. J Pediatr Psychol. 2020;45(4):434–44.
Kazak AE, Hocking MC, Ittenbach RF, Meadows AT, Hobbie W, DeRosa BW, et al. A revision of the Intensity of Treatment Rating Scale: classifying the intensity of pediatric cancer treatment. Pediatr Blood Cancer. 2012;59(1):96–9.
Forrest CB, Bevans KB, Tucker C, Riley AW, Ravens-Sieberer U, Gardner W, et al. The Patient-Reported Outcome Measurement Information System (PROMIS ®) for children and youth: application to pediatric psychology. J Pediatr Psychol. 2012;37(6):614–21.
Knafl K, Deatrick J, Gallo A, Holcombe G, Bakitas M, Dixon J, et al. The analysis and interpretation of cognitive interviews for instrument development. Res Nurs Health. 2007;30(2):224–34.
Willis G, Boeije H. Reflections on the Cognitive Interviewing Reporting Framework: efficacy, expectations, and promise for the future. Methodology. 2013;9(3):123–8. https://doi.org/10.1027/1614-2241/a000074.
Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277–88.
Elo S, Kääriäinen M, Kanste O, Pölkki T, Utriainen K, Kyngäs H. Qualitative content analysis: a focus on trustworthiness. SAGE Open. 2014;4(1):1–10.
Glaser BG. Basics of grounded theory analysis: emergence vs. forcing. Mill Valley, CA: Sociology Press; 1992.
Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57. https://doi.org/10.1093/intqhc/mzm042.
Levitt HM, Bamberg M, Creswell JW, Frost DM, Josselson R, Suárez-Orozco C. Journal article reporting standards for qualitative primary, qualitative meta-analytic, and mixed methods research in psychology: the APA Publications and Communications Board Task Force Report. Am Psychol. 2018;73(1):26–46. https://doi.org/10.1037/amp0000151.
Wu YP, Thompson D, Aroian KJ, McQuaid EL, Deatrick JA. Commentary: writing and evaluating qualitative research reports. J Pediatr Psychol. 2016;41(5):493-505. doi:1093/jpepsy/jsw0321.
Creswell JW. Klassen AC. Smith KC. Best practices for mixed methods research in the health sciences. NIH Office of Behavioral and Social Sciences Research: Plano Clark VL; 2011. https://obssr.od.nih.gov/wp-content/uploads/2016/02/Best_Practices_for_Mixed_Methods_Research.pdf.
McGraw KO, Wong SP. Forming inferences about some intraclass correlation coefficients. Psychol Methods. 1996;1(1):30–46. https://doi.org/10.1037/1082-989X.1.30.
Forman-Hoffman VL, Middleton JC, McKeeman JL, Stambaugh LF, Christian RB, Gaynes BN, et al. Quality improvement, implementation and dissemination strategies to improve mental health care for children and adolescents: a systematic review. Implement Sci. 2017;12:93.
Hedges LV, Rhoads CH. Correcting an analysis of variance for clustering. Br J Math Stat Psychol. 2011;64(Pt 1):20–37.
Wears RL. Advanced statistics: statistical methods for analyzing cluster and cluster-randomized data. Acad Emerg Med. 2002;9(4):330–41.
Hoyle RH, editor. Structural equation modeling: concepts, issues, and applications. Thousand Oaks: Sage; 1995.
Ullman JB, Bentler PM. Structural equation modeling. In: Weiner IB, editor. Handbook of psychology. John Wiley & Sons, Inc.; 2003. https://doi.org/10.1002/0471264385.wei0224.
Byrne BM, Crombie G. Modeling and testing change: an introduction to the latent growth curve model. Understand Stat. 2003;2(3):177–203.
DeLucia C, Pitts SC. Applications of individual growth curve modeling for pediatric psychology research. J Pediatr Psychol. 2005;31(10):1002–23.
Duncan TE, Duncan SC, Strycker LA. An introduction to latent variable growth curve modeling: concepts, issues, and applications. 2nd ed. Mahwah, NJ: Lawrence Erlbaum Associates Publishers; 2006.
Muthen LK, Muthen BO. Mplus user’s guide. 5th ed. Muthen & Muthen: Los Angeles, CA; 2009.
Patrick DL, Burke LB, Gwaltney CJ, Leidy NK, Martin ML, Molsen E, et al. Content validity - establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO Good Research Practices Task Force Report: part 2--assessing respondent understanding. Value Health. 2011;14(8):978–88.
This research was funded by a grant from the American Cancer Society (RSG-19-122). Additional support for this research was provided by the Center for Pediatric Traumatic Stress (SM080048), the Nemours Center for Healthcare Delivery Science, the Nemours Center for Cancer and Blood Disorders, and the Department of Pediatrics at the Children’s Hospital of Philadelphia.
Ethics approval and consent to participate
The Institutional Review Board (IRB) of the Children’s Hospital of Philadelphia serves as the IRB of record for this research and the single IRB for the trial. The protocol was reviewed and approved, with an exemption for aim 1 (CHOP IRB 19-016806[19B0137]) and expedited review for aim 2 (CHOP IRB 19-017117). We will review any needed human subject consideration for aim 3 with the CHOP IRB when the dissemination materials are finalized after the conclusion of the randomized trial.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This protocol follows reporting standards using the Standards for Reporting Implementation Studies (StaRI). The StaRI checklist has been uploaded as a supplemental file. The completed CONSORT checklist for cluster-randomized trials is also uploaded as a supplemental file.
About this article
Cite this article
Kazak, A.E., Deatrick, J.A., Scialla, M.A. et al. Implementation of family psychosocial risk assessment in pediatric cancer with the Psychosocial Assessment Tool (PAT): study protocol for a cluster-randomized comparative effectiveness trial. Implementation Sci 15, 60 (2020). https://doi.org/10.1186/s13012-020-01023-w