Skip to main content

Table 2 Methods of economic-focused data collection and analysis by phase of implementation

From: Extending analytic methods for economic evaluation in implementation science

  Exploration Preparation Implementation Sustainment
Economic Evaluation Methods Selection of interventions and consideration of what implementation requires using prior literature and examples Selection and tailoring of implementation strategies and the creation of tools to support planned intervention implementation and to guide subsequent data collection Data collection and ongoing evaluation of intervention implementation Evaluation of implementation and effectiveness outcomes and planning for local sustainment and/or scale-up/spread of the intervention
Stakeholder interviews, surveys, and periodic reflections • Identify key stakeholders and assess their readiness for an intervention under consideration
• Review existing studies on personnel time and costs associated with intervention implementation
• Solicit feedback on potential barriers to implementation of the intervention
• Assess capacity and stakeholders’ goals and preferences
• Determine which activities and types of costs are associated with usual care
• Design guides to evaluate costs and time from the perspective of those implementing the intervention
• Identify and prepare for potential challenges in intervention implementation
• Obtain feedback on the successes and challenges of implementation
• Understand unexpected challenges (e.g., turnover) in intervention implementation
• Identify opportunities to better support agents involved in implementing the intervention
• Assess how usual care and associated costs may change over time as a result of intervention implementation
• Assess stakeholder buy-in for sustainment of the intervention and necessary adaptations
• Understand unexpected challenges (e.g., turnover) in intervention implementation
Patient and caregiver interviews, surveys, and focus groups • Review existing studies on patient/caregiver preferences, costs, etc. when designing the intervention and identifying relevant costs • Design guides to evaluate costs and time from the patient/caregiver perspective • Conduct interviews/surveys to collect patient/caregiver costs and time spent
• Refine implementation strategies to better address the needs of patients/caregivers and reduce burden
• Include patient/caregiver-level costs in societal perspective for cost-effectiveness analyses
Process flow diagramming • Review process flow diagrams identified in the literature and compare documented processes to local context
• Identify potential resource points
• Consider which personnel are needed and what types of training they need
• Identify potential adaptations to implementation given local challenges, processes in place, and available resources
• Assess key processes involved that affect costs before doing microcosting, cost-effectiveness analysis, etc. through developing a local process flow diagram
• Identify all resource points
• Consider which personnel are needed and what types of training they need
• Obtain feedback on the diagram from diverse implementation agents to ensure the included steps are comprehensive and reflective of their activities
• Map cost data collection activities, including type, source, and frequency, onto the steps to guide methods for estimating costs
• Identify points to assess fidelity using the steps outlined in the diagram
• Track costs and other resources associated with each step involved in implementing the intervention
• Identify and measure potential voltage drops (i.e., retention, missed opportunities, etc.) in implementation reach and desired outcomes
• Revise diagram as needed to reflect changes and adaptations to the intervention and how it is implemented
• Include costs from all steps documented in the diagram in cost-effectiveness analyses
• Adapt processes to address voltage drops and increase effectiveness in the local context
• Identify resource-intensive process steps and consider opportunities for change
• Track and evaluate costs associated with sustainment of the intervention
Time-and-motion analysis • Obtain preliminary list of cost components from existing studies and consider feasibility in the local context • Harmonize cost measures
• Design time-and-motion study, including the creation of toolkits for tracking costs, using process flow diagrams
• Pilot time-and-motion toolkits and solicit feedback from agents involved in implementation
• Observe processes and record activities
• Assess at multiple time points to consider improved efficiencies over time, differences across staff, etc.
• Use time-and-motion data in cost-effectiveness and simulation studies
• Determine how different inputs affect overall cost-effectiveness
• Optimize how processes are performed
• Estimate budget for those interested in program implementation in other settings
System support mapping • Review existing studies on resources and staffing required for intervention implementation
• Identify the individuals responsible for implementing the intervention in the local context
• Identify the individuals responsible for implementing the intervention in the local context
• Develop guide for facilitating system support mapping sessions
• Identify roles and responsibilities of staff implementing the intervention and the resource requirements associated with those responsibilities
• Consider contextual factors that may affect resource adequacy and delineation of roles
• Adapt intervention as needed to address issues related to resource adequacy in the local context
• Adapt intervention as needed to address issues related to resource adequacy in the local context
• Determine the essential activities and resources needed to implement the intervention to support scale-up and spread
Simulation • Review prior simulation studies to motivate the implementation of an intervention
• Develop and continue to update a list of the intervention costs and effectiveness estimates from literature review
• Identify potential gaps in the types of costs associated with implementation of the intervention reported in prior studies
• Identify short-term and long-term outcomes of interest
• Document all decisions made about how the intervention is implemented
• Estimate the costs and other resources required to implement the intervention using tools developed (e.g., process flow diagrams, time-and-motion toolkits, etc.)
• Estimate the costs and other resources required to implement the intervention using tools developed (e.g., process flow diagrams, time-and-motion toolkits, etc.)
• Develop and refine model that accounts for all aspects of intervention implementation in the local context
• Estimate population-level projections of the impact of the intervention for the local context
• Compare expected outcomes in the local context for the overall population and by subgroups to explore potential equity issues
• Develop and refine model that accounts for all aspects of intervention implementation in the local context
• Estimate population-level projections of the impact of the intervention for the local context
• Compare expected outcomes in the local context for the overall population and by subgroups to explore potential equity issues
• Estimate the potential cost-effectiveness of the intervention if sustained or scaled up
• Determine optimal strategies for intervention sustainment/scale-up of the intervention
Sensitivity/Uncertainty analysis • Identify important parameters in prior estimates of intervention effectiveness and costs
• Identify areas of uncertainty related to the intervention and implementation of the intervention in prior simulation studies
• Identify important parameters in prior estimates of intervention effectiveness and costs
• Identify areas of uncertainty related to the intervention and implementation of the intervention in prior simulation studies
• Identify areas of uncertainty while planning implementation of the intervention in the local context
• Identify areas of uncertainty while planning implementation of the intervention in the local context
• Track variations in the time spent and costs associated with intervention implementation to inform best and worst case scenario estimates
• Determine the key parameters and assumptions that have the largest influence on the results
• Evaluate the impact of uncertainty on outcomes of interest in the local context
• Estimate the sensitivity of the simulation results to changes in intervention implementation in the local context
• Graphically display the results of sensitivity and uncertainty analyses to inform decision-making
• Determine the key parameters and assumptions that have the largest influence on the results
• Evaluate the impact of uncertainty on outcomes of interest in the local context
• Estimate the sensitivity of the simulation results to changes in intervention implementation in the local context
• Estimate and report plausible ranges of outcomes of interest if the intervention is sustained or scaled up
• Graphically display the results of sensitivity and uncertainty analyses to inform decision-making