Table 2 Implementation science challenges and artificial intelligence solutions with caveats

Speed

 Qualitative partner engagement methods require personnel time to conduct

AI chatbots could be trained to moderate/lead

AI needs to be monitored to ensure prompts and questions are appropriate, responsive, and unbiased

 Quantitative and qualitative data collection, analyses, and iterations require time

NLP/text mining can automate and speed up the process of:

• Qualitative analyses

• Collecting unstructured contextual data documented in electronic health records and other data sources

AI algorithms can automate iterative data collection and analyses and display findings in accessible ways (e.g., dashboards)

AI can replicate analyses and predictions with different subsamples when have sufficiently big data

AI needs to be monitored to ensure appropriate, complete, accurate, and replicable analyses, including for sentiment and inherent biases in data, as well as data drift

 Intervention adoption is often slow due to difficulty testing implementation strategies that achieve behavior change and lead to adoption

With big data, AI can analyze some types of implementation strategies in almost real-time

NLP and AI-enabled chatbots can create tailored messaging or nudges for clusters based on the characteristics of the intended adopters

Inherent ethical tensions with the use of nudges that should be carefully considered (e.g., ensure preserving autonomy and promoting positive change)

Sustainability

 Iterative evaluation of the dynamically changing context is important to improve sustainability but challenging to achieve given the time and effort required

Predictive AI algorithms can assist in assessing the chance of sustainment

AI algorithms can continually work in the background to assess changes (e.g., dashboards)

NLP can make ongoing/iterative qualitative assessment more efficient and feasible

AI needs to be monitored for data drift and biases in data

AI needs to be monitored to ensure appropriate, complete, and accurate analyses, including for sentiment

Equity

 Language, digital literacy, and social needs can be barriers to participating in:

  • Partner engagement

  • Implementation studies

  • Intervention education implementation strategies

AI-enhanced translation software (i.e., Google Translate) can increase access to and inclusion

Patients with these characteristics are often less likely to be part of AI data sets- (e.g., genomic registries, electronic health record databases)

Messages through translation software may be distorted, mistrusted, or insensitive/off-putting

 People from historically marginalized groups may not respond to risk communication, behavior change, or decision-making messaging that is effective in other groups or experience mistrust

Culturally tailored AI-enabled messaging or chatbots can support the adoption of EBI in historically marginalized groups

AI-enabled chatbots may be able to improve racial empathy gaps and provide feedback on issues such as social determinants, and risk status [32]

Use of nudges could have ethical implications if not monitored or used responsibly

Could alienate groups further if not done well or relying on superficial “surface” disparity information

Could perpetuate distrust of the medical system and lead to distrust regarding the purpose of data collection

Inaccurate predictions could result from biased or missing data

 Historically marginalized groups may experience large disparities and are often under-represented in surveillance systems, surveys, and other data sources

AI can expand data sources and recruit participants through various online and other channels to increase the breadth of representativeness

Could worsen the underrepresentation of certain groups and study conclusions if not carefully considered

Generalizability

 Scope/reach of partner engagement may be limited or may not reach “right” people due to time and manpower required to conduct engagement methods

AI can recruit participants through various online channels, improving the breadth of perspectives and data quality while greatly reducing the human power required

AI chatbots could be trained to moderate/lead or generate agendas and recruitment materials (e.g., ChatGPT)

Need to monitor for new biases, inequities or unintended consequences introduced or created due to:

• irrelevant or partial samples of populations

• analyses of biased data

AI needs to be monitored to ensure prompts and questions are appropriate, responsive, and unbiased

 Surveys and other traditional D&I data sources may be limited to partial or small samples and represent limited partner perspectives

AI can utilize social media and other secondary data sources, including observational data to glean valuable insights about population trends and public sentiment

AI can generate synthetic datasets that closely mimic real-world data sources. These synthetic datasets, created based on patterns and structures found in real data, can supplement existing survey data, thereby broadening the scope of analysis without incurring additional data collection costs

AI can deploy surveys effectively through various online channels, potentially improving response rates and data quality while reducing the human power required

AI can assess sustainment of generalizability over time with continued vigilance post-intervention

Without monitoring new biases, may lead to inequities or unintended consequences introduced or created due to:

• inaccurate sentiment analyses

• incomplete population samples

• inclusion of biased data

• generation of biased data sets

May lead to a lack of transparency when developing synthetic datasets and considering regulations 

Data may not be available for AI to assess for key issues, such as trust or racism

Assessing context and context-outcome relationships

 Traditional approaches to understanding context/determinants is limited to the “stated” (e.g., qualitative partner engagement, surveys) and the “realized” (e.g., quantitative data sources) that require an a priori signal and are usually cross-sectional evaluations

AI algorithms can digest large amounts of quantitative data, including from non-traditional data sources (e.g., wearables), and look for patterns in contextual variables and de novo signals, allowing for the identification of issues/problems/inequities that we didn’t know to look for

Data sources could include observations of activities or processes (e.g., time stamps or clicks within the electronic health record) to extend contextual data to assess

Can replicate AI algorithms to look for stability over time or in different settings or contextual situations

Need to consider implications of “black box” algorithms and whether “explainability” and validation or further exploration is needed—consider regulations and recommendations

 Measures of organizational culture are important for implementation but are underdeveloped and seldom available on large and representative samples

NLP with sentiment analyses can be applied to data sources such as emails, text messages, and videos to measure culture

Monitoring is invasive to members of the organization and presents new ethical considerations

Assessing causality and mechanisms

 Quantitative analyses in D&I are often descriptive, are rarely comparative, and do not account for complex and non-linear interactions between context/determinants or outcomes, which limits causal assessments of outcomes under specific contextual situations for different:

  • IS TMFs used

  • Interventions

  • Implementation strategies

AI algorithms can account for the intangible interactions between data, leading to new insights

AI algorithms can compare outcomes of exposure groups using complex and large data sets

When examining the multilevel context in a D&I study, AI may allow us to examine interactions and effects across levels

Need to consider implications of “black box” algorithms and whether “explainability” and validation or further exploration is needed—consider regulations and recommendations