Skip to main content


Table 3 Main findings obtained from appraising the applicability of the framework

From: Making sense of complexity in context and implementation: the Context and Implementation of Complex Interventions (CICI) framework

Internal coherence and completeness of framework
Coherence In terms of coherence, the framework was appraised as largely consistent; significant overlaps between the context and implementation dimensions were noted. In the qualitative review, issues arose in relation to the attribution of text extracts to specific domains (e.g., provider vs. organisational domain). Also, data extractors were not comfortable with our original definition of the ethical domain.
Completeness In relation to completeness, researchers considered the framework complete at the level of the domains but not with respect to the aspects covered. Consequently, missing aspects (e.g., health care system) were added. In the applicability assessment, the framework partly missed the complexity and adaptability of the intervention [14]. In particular, the concept of reinforced home-based palliative care, as defined for the HTA, did not fit the German context, where reinforcement (e.g., carer support) is integrated in every palliative care construct available.
Theory advancement and development
Theory While its generic version does not display relationships, the framework allows for their assessment when applied to a complex intervention. The framework was also considered helpful in guiding the formulation of questions about these links, which in return inspires the advancement of theory regarding interactions between domains as well as context and implementation dimensions.
Compatibility The framework was assessed as compatible with other methods developed and applied within INTEGRATE-HTA (i.e., assessments of ethical or socio-cultural issues, logic model and the INTEGRATE-HTA model) as a consequence of their development taking place side-by-side. As the framework builds on a systematic review of previously published conceptual frameworks, theories and models of context and implementation, it can also be considered compatible with the literature.
Relationships In line with the tradition of previous implementation frameworks, the application of the revised CICI framework in quantitative reviews does not make relationships between dimensions or domains explicit. On the other hand, in the qualitative systematic review, the framework facilitated the assessment of relationships through data extracts being attributed to several domains, for example, ‘access to healthcare’ emerged as a relevant aspect under three context domains (geographical, political, ethical and socio-economic). Moreover, the graphical display arising from its use highlights how dimensions and domains interact within a complex system.
Applicability of framework
Adaptation and applicability A certain degree of adaptation with regard to the domains to be considered in relation to a specific intervention is clearly a strength of the framework. However, to avoid ‘cherry-picking,’ the decision of which domains to consider should be a structured and transparent process. The successful application of the framework to very different types of health interventions suggests that the criterion applicability is met. While studies looking at policy and environmental interventions reported more details on the macro context, studies evaluating educational, psychoeducational, psychological, social as well as spiritual interventions reported more details on the meso level relating to context or setting.
Flexibility Generally, the framework has proven to be methodologically flexible due to its good applicability in quantitative and qualitative systematic reviews and the primary qualitative, quantitative or mixed-method studies included in these, although data extraction was sometimes limited by poor reporting. The framework also showed flexibility in relation to the demonstration HTA as a whole and the applicability assessment.
Capability The high granularity of the framework proved capable of facilitating extraction of—often scarcely—reported information on context and implementation and supported the structuring of this information. It therefore helps to review all sources of complexity in relation to a given intervention and to examine the interactions between them.
User-friendliness of framework
Feasibility of application The framework was considered feasible to apply across all the different applications; ease of use was ensured through clear use of terminology, step-by-step guidance and the provision of ready-to-use data extraction tools. In quantitative reviews, extracting data on context and implementation imposes an additional workload on researchers. The additional time required is limited, where extraction is only performed on the included study; it may be substantial if further sources (e.g., qualitative studies or process evaluations cited in the included study) are also consulted.