Skip to main content
Download PDF
Download ePub

Testing the organizational theory of innovation implementation effectiveness in a community pharmacy medication management program: a hurdle regression analysis

Implementation Science volume 13, Article number: 105 (2018) Cite this article

Abstract

Background

Many state Medicaid programs are implementing pharmacist-led medication management programs to improve outcomes for high-risk beneficiaries. There are a limited number of studies examining implementation of these programs, making it difficult to assess why program outcomes might vary across organizations. To address this, we tested the applicability of the organizational theory of innovation implementation effectiveness to examine implementation of a community pharmacy Medicaid medication management program.

Methods

We used a hurdle regression model to examine whether organizational determinants, such as implementation climate and innovation-values fit, were associated with effective implementation. We defined effective implementation in two ways: implementation versus non-implementation and program reach (i.e., the proportion of the target population that received the intervention). Data sources included an implementation survey administered to participating community pharmacies and administrative data.

Results

The findings suggest that implementation climate is positively and significantly associated with implementation versus non-implementation (AME = 2.65, p < 0.001) and with program reach (AME = 5.05, p = 0.001). Similarly, the results suggest that innovation-values fit is positively and significantly associated with implementation (AME = 2.17, p = 0.037) and program reach (AME = 11.79, p < 0.001). Some structural characteristics, such as having a clinical pharmacist on staff, were significant predictors of implementation and program reach whereas other characteristics, such as pharmacy type or prescription volume, were not.

Conclusions

Our study supported the use of the organizational theory of innovation implementation effectiveness to identify organizational determinants that are associated with effective implementation (e.g., implementation climate and innovation-values fit). Unlike broader environmental factors or structural characteristics (e.g., pharmacy type), implementation climate and innovation-values fit are modifiable factors and can be targeted through intervention—a finding that is important for community pharmacy practice. Additional research is needed to determine what implementation strategies can be used by community pharmacy leaders and practitioners to develop a positive implementation climate and innovation-values fit for medication management programs.

Peer Review reports

Background

Many state Medicaid programs have expanded enrollment eligibility under the Affordable Care Act, making Medicaid the largest health insurance program in the USA [1, 2]. Medicaid spending is largely driven by a small subset of high-risk patients; 5 % of Medicaid beneficiaries account for almost half of Medicaid expenditures [3]. This small subset of beneficiaries is disproportionately impacted by chronic conditions, such as diabetes and asthma, and the co-occurrence of difficult-to-treat conditions (e.g., substance use and mental health conditions) [3]. To improve chronic disease management, several Medicaid programs have implemented medication management programs in partnership with pharmacists [4,5,6].

Pharmacist-led medication management programs have improved patients’ medication adherence and therapeutic outcomes (e.g., blood pressure, hemoglobin A1C) while reducing healthcare costs [7,8,9,10]. However, researchers have had difficulty attributing changes in patient outcomes to specific program features due to the wide variability in medication management programs [4, 11]. In the Medicare Part D Medication Therapy Management (MTM) program, for example, researchers have noted that medication services are delivered in a variety of settings (e.g., call centers, outpatient care) and formats (e.g., in-person vs. phone) [11]. Similar challenges exist in Medicaid medication management programs—programs vary in patient eligibility criteria, the services provided, and the setting of service delivery [4].

In addition to program design variability, there are a limited number of studies examining implementation of pharmacist-led medication management programs, making it difficult to assess why program outcomes might vary across organizations. Many of the studies that have examined organizational determinants of implementation effectiveness in pharmacist-led medication management programs have been qualitative, limiting their generalizability, or have not been guided by a theory, making it difficult to interpret the findings. Past studies have identified factors, such as organizational structure (e.g., staff size), leadership support, and financial resource availability [12,13,14,15,16], but not applied theory to demonstrate how these factors work in concert to produce effective implementation. Thus, this study will test the applicability of the organizational theory of innovation implementation effectiveness to examine implementation of a community pharmacy Medicaid medication management program.

Conceptual framework

Implementation theories have been developed to identify the organizational factors and underlying relationships that are hypothesized to influence effective implementation (i.e., the quality and consistency of implementation) [17, 18]. The organizational theory of innovation implementation effectiveness was designed for complex innovations like medication management programs, which often require coordinated use by multiple individuals to be effective [18,19,20]. This theory posits that effective implementation is driven by an organization’s implementation climate and the fit between the innovation and organization values (Fig. 1) [18,19,20]. For example, a community pharmacy might develop formal policies to support implementation of a medication management program such as employee training or reward and recognition systems. The collective influence of the pharmacy’s implementation policies, in turn, affects employees’ shared perceptions about the extent to which the medication management program is rewarded, supported, and expected (implementation climate) [18,19,20]. Positive perceptions about implementation climate are likely to increase employees’ acceptance of medication management programs, increasing the likelihood that pharmacy staff will appropriately implement medication management programs (i.e., as the pharmacy intended for it to be implemented) and ultimately increase implementation effectiveness. Therefore, we hypothesize that positive perceptions about implementation climate will be positively associated with implementation effectiveness (H1).

Fig. 1
figure 1

The impact of implementation climate and innovation-values fit on implementation effectiveness

Full size image

The organizational theory of innovation implementation effectiveness maintains that innovation-values fit also affects implementation effectiveness [18,19,20]. In this specific case, innovation-values fit refers to pharmacy employees’ perceptions about how well medication management programs align with the values of the pharmacy and the pharmacy profession. Innovation-values fit is believed to affect implementation effectiveness both directly and indirectly. If pharmacy employees perceive that medication management programs do not align with the pharmacy’s values, the employee may be less committed to implementation and exert less effort towards ensuring effective implementation [18]. Innovation-values fit is also likely to impact the relationship between implementation climate and implementation effectiveness [18,19,20]. Since innovation-values fit affects commitment, the impact of implementation climate on implementation effectiveness will be amplified by positive, and weakened by negative, perceptions about innovation-values fit. Thus, we hypothesize that positive perceptions about innovation-values fit will directly and positively affect implementation effectiveness (H2) and moderate the relationship between implementation climate and implementation effectiveness (H3).

Implementation effectiveness is also likely to be affected by broader environmental and organizational factors (e.g., organizational context), such as patient needs and resources, available resources, access to knowledge about the intervention, and structural characteristics [21]. For example, community pharmacies that serve a higher proportion of high-risk patients may be better at implementing innovations for high-risk populations. Additionally, pharmacies in rural locations may be better at implementing innovations for high-risk populations since residents in rural areas have higher rates of chronic illness [22]. Implementation effectiveness is also likely to be positively influenced by a pharmacy’s available resources, such as amount of staff and training of staff, and access to knowledge about medication management programs (e.g., experience implementing similar interventions). Conversely, certain structural characteristics may negatively affect implementation effectiveness. For example, pharmacies that have opened recently may not have as strong of ties with patients as pharmacies that have been in operation for many years (e.g., the liability of newness hypothesis) [23]. Similarly, larger pharmacies may be impeded by a more formal organizational structure, which can negatively impact innovation implementation [24]. For instance, managers of independently owned pharmacies may have greater decisional autonomy because there is less formalization in the organization and, as a result, be better able to support implementation of medication management programs. Therefore, we hypothesize that four contextual factors, patient needs and resources, available resources, access to knowledge about the intervention, and structural characteristics, will affect implementation effectiveness (H4).

Methods

Study design

We used a cross-sectional design examining implementation of a community pharmacy Medicaid medication management program during the program year of 2016. The unit of analysis was at the pharmacy level.

Intervention description

The community pharmacy enhanced services network (CPESNSM) is a national network of community pharmacies that offer medication management services [25]. This study examines the North Carolina CPESN (NC-CPESN), the pioneer site for CPESN [6]. NC-CPESN was launched in 2014 by the Community Care of North Carolina (CCNC)—the primary case management provider for NC Medicaid beneficiaries [6, 26]. NC-CPESN is voluntary and allows any community pharmacy if the following requirements are met: (1) provide certain medication management services, (2) be responsible for the outcomes of a defined patient population through value-based payment, and (3) tailor services based on patients’ risk score. One of the key services required for reimbursement is a comprehensive medication review (CMR) to identify opportunities for improving medication management and reduce risk of medication problems. Patients include Medicaid, Medicare, and NC Health Choice beneficiaries, as well as dual-eligible patients. The payment model is a per-member per-month payment model based on patient’s risk score (described below) and the pharmacy’s performance score on a series of measures including medication adherence, total cost of care, hospital admission rate, and emergency department admissions [27].

Study population

The study population included community pharmacies that participated in either the first year or the second year of the 3-year NC-CPESN program (September 2014–August 2017); pharmacies that joined in the third year were excluded from the analysis because they had little-to-no experience with implementation at the time of the survey (described below).

Data sources

In fall 2016, we administered a paper-based survey to community pharmacies that participated in either the first or second year of the NC-CPESN program. The survey assessed pharmacies’ structural characteristics, experience with NC-CPESN, and perceptions about implementation (e.g., implementation climate, innovation-values fit). A copy of the survey has been published elsewhere [28]. A committee of researchers and community pharmacy practitioners (n = 25) reviewed the survey items’ content, readability, and formatting. The survey questions were also piloted in a small group of community pharmacists (n = 5) who were identified as experts by the committee based on their job tenure and reputation in the field of community pharmacy. From the committee review and the initial pilot test, we received similar types of feedback and felt that further pilot testing was not needed. The survey was mailed to participating pharmacies along with other NC-CPESN program materials to increase the response rate. Pharmacies also received three email reminders at ~ 2, 4, and 8 weeks after the survey was mailed. Within the pharmacy, employees that were intended users of NC-CPESN (e.g., pharmacists, pharmacy technicians, and administrative staff) completed the survey. We did not include supporters of NC-CPESN (e.g., pharmacy owners who supported the intervention through policies and resources but did not directly implement the intervention) since their actions indirectly rather than directly affect implementation, a decision that is consistent with other implementation studies [19, 29]. We had more than one respondent per pharmacy; therefore, the responses were aggregated to the pharmacy level (description below). We received surveys from 191 of 268 pharmacies (71.3% response rate). Participants provided written consent. The Institutional Review Board of the University of North Carolina at Chapel Hill approved this study (IRB # 17-1304).

In addition, we used 2016 NC-CPESN program administrative data and 2016 county health ranking data [30]. Program administrative data provided information on the number of high-risk patients attributed to each pharmacy, patient demographics, and the number of CMRs that were delivered. County health ranking data included county-level measures of clinical (e.g., healthcare access) and social (e.g., insurance status) factors that might affect the pharmacy’s implementation of NC-CPESN [30]. The operationalization of these measures is described below.

Dependent variables

We used two variables to measure implementation effectiveness—one indicator for implementation versus non-implementation and one indicator for program reach.

Implementation of a CMR for high-risk patients

Based upon whether a pharmacy implemented a CMR on any attributed high-risk patient, we divided the sample into implementers (e.g., ≥ 1 CMR for an attributed high-risk patient) and non-implementers (e.g., no CMR for any attributed high-risk patients) during the program quarter Nov. 2016–Jan. 2017. We chose this quarter because there were no changes to the intervention (e.g., intervention requirements or payment model) during this or the previous quarter. High risk was defined as having a care triage score ≥ 75. Care triage score is a proprietary measure used by CCNC to estimate a patient’s risk for hospitalization and includes variables such as the number of chronic conditions a patient has and the type of medication the patient is taking. Patients with care triage scores > 75 are considered a priority population for CCNC. Patients are defined as attributed to a pharmacy if they filled at least one chronic medication within the last 90 days and ≥ 80% of their medications for at least two of 3 months within the quarter. Patients also had to be eligible for Medicaid or Medicare for at least two of the 3 months within the quarter. The patient attribution process is done on a monthly basis. On average, in any given month, about 100,000 NC residents are attributed to NC-CPESN pharmacies [27].

Proportion of high-risk patients receiving a CMR

We measured implementation effectiveness to assess the reach of the intervention among attributed high-risk patients. We calculated the number of attributed high-risk patients receiving a CMR divided by the number of high-risk, attributed patients per pharmacy during the program quarter Nov. 2016–Jan. 2017.

Independent variables

Implementation climate

Implementation climate was defined using four survey items assessing the extent to which NC-CPESN was supported, rewarded, and expected within the pharmacy (e.g., “Our pharmacy allocates sufficient time to delivering enhanced pharmacy services” and “Our pharmacy devotes adequate resources to implementing enhanced pharmacy services”) [18,19,20]. The survey items were adapted for a pharmacy setting from a scale validated in an oncology setting [31, 32]. The questions included group rather than individual referents, which is recommended when assessing organizational-level outcomes such as implementation climate [29]. Each item was measured on 5-point Likert scale ranging from 0 (strongly disagree) to 4 (strongly agree). The survey items were summed for individual staff members who worked directly on implementation (i.e., innovation users), and a mean was calculated to produce a pharmacy-level measure. Higher values of the score corresponded with positive perceptions of implementation climate.

Innovation-values fit

Innovation-values fit was defined using four survey items assessing staff perceptions about the extent to which NC-CPESN fit with the values of the pharmacy (e.g., “Delivering enhanced pharmacy services is consistent with providing the best care possible for our patients”) and of the pharmacy profession (“Delivering enhanced pharmacy services is important for advancing the field of pharmacy”) [18,19,20]. To identify “high-intensity” values, or values that are highly important to pharmacy staff, we obtained pharmacy practitioner input during the survey pilot (described above) [18,19,20]. For example, practitioners described the importance of improving the quality of services across all community pharmacies, not just the quality of services within their own pharmacy. As a result, practitioners described valuing interventions that would advance the field of community pharmacy as a whole. To address this, we included a survey item to assess perceptions about whether NC-CPESN was advancing the field of pharmacy. As with implementation climate, the innovation-values fit questions were group-referenced, measured on the same 5-point Likert scale, aggregated from individual responses to produce a pharmacy-level mean, and ordered so that higher scores corresponded to positive perceptions.

Other independent variables

Patients’ needs and resources were measured by rural location, clinical factors, social factors, 340B participation, and proportion of high-risk patients. Rural location was defined as a binary variable (e.g., urban, rural) using a zip code approximation of the rural-urban commuting area codes. Clinical factors were defined using a pre-existing, county-level composite measure of access to care items (e.g., primary care provider ratio, uninsured rate) and quality of care items (e.g., preventable hospital stays, diabetes monitoring) ranging from 0 to 100 [30]. Social factors were defined using a pre-existing, county-level composite measure of items such as education, employment, uninsured, and income ranging from 0 to 100 [30]. The clinical and social factor scales were recoded so that higher values on the scale were associated with better patient outcomes. Participation in 340B Drug Pricing Program was measured as a binary variable. The 340B Drug Pricing Program is a federal program that requires drug manufacturers to provide outpatient drugs to eligible healthcare organizations (e.g., safety net providers) at a discounted rate [33]. Community pharmacies can participate in this program by dispensing 340B drugs through a contract with eligible healthcare organizations. Proportion of high-risk patients was defined as the number of attributed high-risk patients divided by the number of attributed patients per pharmacy over a program quarter.

Available resources

Available resources were measured by three variables: the presence of a clinical pharmacist (binary), total number of full- and part-time staff (e.g., pharmacists, pharmacy technicians, administrative staff), and the presence of pharmacy students or residents in the past month (binary). A clinical pharmacist is defined as a pharmacist whose role focuses not only on dispensing medication but also on the clinical care of patients, such as optimizing patients’ medication regimens and providing health education and preventive health services [34]. In community pharmacies, a clinical pharmacist typically has some or all of their time devoted to activities outside of dispensing medications, such as delivering medication management services [35]. The role of clinical pharmacist, such as the type of clinical services offered within the pharmacy, as well as the amount of time devoted to non-dispensing activities, varies, however, across community pharmacy settings.

Access to knowledge about the intervention

Access to knowledge about the intervention was measured in three ways: (1) experience with NC-CPESN, defined as the number of months the pharmacy was enrolled in NC-CPESN; (2) past performance with NC-CPESN, measured using a lagged dependent variable (e.g., proportion of CMRs completed per high-risk patients) for the previous program quarter (Aug–Oct 2016); and (3) participation in Medicare Part D MTM (binary).

Structural characteristics

Structural characteristics were assessed by three variables. First, independent ownership was a binary variable: single- and multiple- independent pharmacies versus chain, outpatient, and federally qualified health center (FQHC) pharmacies. Independently owned pharmacies include single-independent pharmacies (i.e., an owner owns one pharmacy) and multiple independent pharmacies (i.e., an owner owns multiple pharmacies). Chain pharmacies include pharmacies owned by a publically traded company. Outpatient pharmacies are pharmacies operated by a health system and located within an outpatient healthcare setting (e.g., primary care office), and FQHC pharmacies are operated by and located within a FQHC. We used a binary variable due to small sample sizes within the chain and outpatient pharmacy categories. Second, prescription volume was dichotomized as low (< 2000 prescriptions/week) versus high (≥ 2000 prescriptions/week). The criterion for low volume was selected based on input of community pharmacy practitioners and researchers. Third, established pharmacies were those that had been in operation for more than 20 years. Similarly, the threshold of 20 years as the criterion for established pharmacy was selected based on input from community pharmacy practitioners and researchers.

Statistical analysis

Descriptive statistics

Frequencies and percentages were used to describe the study population. We conducted bivariate analyses to compare the sample characteristics between implementers (completed ≥ 1 CMR during the program quarter for high-risk patients) and non-implementers (no completed CMR during the program quarter for high-risk patients).

Exploratory factor analyses

To determine if implementation climate and innovation-values survey items could be used as distinct variables, we conducted three analyses. First, we examined pairwise correlations among the items and conducted a Bartlett’s test of sphericity and a Kaiser-Meyer-Olkin (KMO) test. Second, we defined the number of initial factors using principal component analysis and rotated the factors using orthogonal varimax rotation to improve interpretability. Finally, we confirmed the number of extracted factors using two decision rules: (1) the number of eigenvalues > 1.0, and (2) the number of eigenvalues from the factor analysis that were larger than the eigenvalues from randomly generated data (e.g., parallel analysis test). We also assessed the internal consistency of the two scales using Cronbach’s coefficient alpha. To ensure the results were not overly sensitive to the method of factor extraction, we conducted a sensitivity analysis by running a common factor analysis using principal axis factoring and did not find differences in the results. We also compared the results of the exploratory factor analyses by staff roles within pharmacies (e.g., pharmacists, pharmacy technicians, and administrative staff) to determine if results from different staff types could be aggregated to the pharmacy level. Factor analyses did not differ by subgroup, suggesting that aggregating subgroups was appropriate.

Hurdle regression model

Hurdle regression is a two-equation model for count data: one equation determines the likelihood of an outcome (e.g., whether a pharmacy implemented a CMR) and the other examines the positive outcomes (e.g., how many CMRs were delivered to high-risk patients) [36, 37]. We used a hurdle regression to model both of these processes and to account for an excess of zeroes in the dependent variable (40.8% of the sample had zero implementation in the program quarter). For the first stage, we used a logistic regression to determine the probability of a pharmacy implementing a CMR for a high-risk patient (e.g., implementer versus non-implementer). For the second stage, we used a zero-truncated negative binomial model to determine how many CMRs were delivered to high-risk patients (e.g., program reach). A negative binomial model was selected over a Poisson model to account for over-dispersion in the data (i.e., the variance was larger than the mean). For the negative binomial model, we treated the denominator (i.e., number of high-risk patients) as the exposure to adjust for differences in the opportunity available to deliver the intervention and assumed the unobserved heterogeneity was gamma distributed (i.e., NB2 model). We compared this model with a zero-inflated negative binomial, which is another two-equation model for count data; we did not find differences in the results. Therefore, we used the hurdle regression.

In the hurdle regression, we included the key variables of interest (e.g., implementation climate, innovation-values fit, and an interaction of the two) and control variables selected a priori (e.g., patient needs and resources, available resources). We assessed the goodness of fit for the interaction term in both stages of the model since interpretation of marginal effects on interaction terms can be complicated in non-linear models [38]. Since the interaction term improved fit in both stages, we included the term. To model the impact of the interaction term, we plotted the marginal effect of innovation-values fit over representative values of implementation climate score for both of the equations. One control variable, past performance with NC-CPESN, was a lagged dependent variable, which can cause biased coefficients if the data generating process is non-stationary [39]. Using the Harris-Tzavalis test [40], which can be used when the number of time periods is small relative to the number of panels, we rejected the null hypothesis that the data generating process is non-stationary. Therefore, we included the lagged dependent variable in the model. We used cluster-robust standard errors to account for clustering that might occur at the network level. NC-CPESN pharmacies are grouped into regional networks by CCNC and may receive different levels and quality of implementation support across networks. All pharmacies received standardized training on how to document a CMR and how to use the documentation system that was required by NC-CPESN; however, the amount and type of technical assistance to support implementation of NC-CPESN (e.g., how to deliver a CMR) varied across networks [41]. Because the amount of missing data in both equations of the model was less than 10% (8.0 and 5.8%, respectively), we addressed missingness using complete case analysis. To test whether missingness might be correlated with the dependent variable, we compared the proportion of implementers and non-implementers between survey respondents and non-respondents and did not find significant differences (\( {\mathcal{X}}^2=2.27 \), p = 0.132). We conducted the analyses using Stata version 13.0 (College Station, TX).

Results

Of the 191 pharmacies in our sample, 113 (59.16%) were implementers. Pharmacies that successfully implemented a CMR had a significantly higher mean implementation climate (11.81 vs. 3.55, p < 0.001) and innovation-values fit (13.55 vs. 11.06, p < 0.001) scores (Table 1). In terms of patient needs and resources, implementing pharmacies were significantly more likely to participate in the 340B Drug Pricing Program (69.12 vs. 30.88%, p = 0.024) and have a higher proportion of high-risk patients (0.42 vs. 0.36, p = 0.004). For available resources, implementing pharmacies were more likely to have a clinical pharmacist (86.49 vs. 13.51%, p < 0.001) and either a pharmacy student or resident on staff (92.86 vs. 7.14%, p < 0.001). Implementing pharmacies had more experience with NC-CPESN (34.37 vs. 27.05 months, p < 0.001) and had a higher proportion of CMRs performed among high-risk patients in the previous quarter (0.03 vs. 0.00, p < 0.001). For structural characteristics of pharmacies, we did not find any significant differences between implementers and non-implementers.

Table 1 Descriptive statistics of community pharmacies participating in NC-CPESN
Full size table

Exploratory factor analysis

All pairwise correlations among the items in the implementation climate and innovation-values scales were greater than 0.30, indicating there was sufficient correlation for factor analysis (Table 2). Further, none of the pairwise correlations exceeded > 0.80, indicating that high multicollinearity was not a problem. The Bartlett’s test of sphericity was significant for the implementation climate (\( {\mathcal{X}}^2=1975.43,p<0.001\Big) \) and the innovation-values fit scale (\( {\mathcal{X}}^2=1077.83,p<0.001\Big) \). Therefore, we rejected the null hypothesis that either matrix was an identity matrix. The KMO statistic for implementation climate and innovation-values fit scales was 0.773 and 0.818, respectively, which are within an acceptable range to support factor analysis (greater than 0.60) [42].

Table 2 Correlation matrix for the implementation climate and innovation-values fit scales
Full size table

Factor loadings produced from the principal component analysis (Table 3) suggest that survey items measuring implementation climate or innovation-values fit load onto two distinct factors. For each set of items, only one factor had an eigenvalue exceeding 1.0 (implementation climate, largest EV = 2.77; innovation-values fit, largest EV = 3.35), and these eigenvalues were greater than eigenvalues from a randomly generated data set, suggesting one factor should be extracted for each set of items. The total amount of variance in the items explained by the two extracted factors was 79.27% for implementation climate and 83.63% for innovation-values fit. There were several items that had double factor loadings (e.g., loaded onto more than one factor) (Table 3); however, based on our decision rules as well as our theory, we retained one extracted factor for each set of items. Cronbach’s coefficient alpha was 0.845 for the implementation climate scale and 0.833 for the innovation-values scale, suggesting the items have “very good” internal consistency [43].

Table 3 Factor loadings from the rotated factor structure matrix for implementation climate and innovation-values fit scales
Full size table

Hurdle regression: equation 1

Hypothesis 1

The first equation of the hurdle regression indicated that a one-unit increase in the implementation climate score increased the probability of NC-CPESN implementation by 2.65 percentage points holding all else constant (p < 0.001) (Table 4). The predicted probability of NC-CPESN implementation for pharmacies with the median implementation climate score (9.14) was 0.66 compared to 0.84 for pharmacies with an implementation climate score at the 75th percentile (12.50).

Table 4 Parameter estimates from hurdle regression of NC-CPESN implementation and program reach of NC-CPESN implementation
Full size table

Hypothesis 2

Similarly, an increase in innovation-values fit score increased the probability of NC-CPESN implementation by 2.17 percentage points (p = 0.037). The predicted probability of NC-CPESN implementation for pharmacies with the median innovation-values score (13.07) was 0.61 compared to 0.66 for pharmacies with an implementation climate score at the 75th percentile (14.68).

Hypothesis 3

The marginal effect of innovation-values fit on the probability of NC-CPESN implementation increased as implementation climate score increased. The marginal effect began to decline at an implementation score of 8 (Fig. 2).

Fig. 2
figure 2

Plot of marginal effect of innovation-values fit and implementation climate score for Equation 1

Full size image

Hypothesis 4

No significant differences in the probability of NC-CPESN implementation was found based on patients’ needs and resources. For available resources, the probability of implementing NC-CPESN was 9.86 percentage points higher for pharmacies that had a clinical pharmacist (p = 0.038). In terms of access to knowledge about the intervention and available resources, amount of experience with NC-CPESN (p = 0.004), past performance with NC-CPESN (p < 0.001), and participation in Medicare Part D MTM (p = 0.003) were each positively associated with the probability of implementing NC-CPESN. Within structural characteristics, the probability of implementing NC-CPESN was 4.14 percentage points higher among independently owned pharmacies (p = 0.041).

Hurdle regression: equation 2

Hypothesis 1

Findings from the second equation of the hurdle regression indicated that a one-unit increase an implementation climate score was associated with a 5.05 increase in implementation of CMRs per high-risk patients holding all else constant (p = 0.001). The predicted number of CMRs per high-risk patients for pharmacies with the median implementation climate score (9.14) was 16.21 compared to 28.10 for pharmacies with an implementation climate score at the 75th percentile (12.50).

Hypothesis 2

Similarly, implementation of CMRs per high-risk patients was positively associated with innovation-values fit score (p < 0.001). The predicted number of CMRs per high-risk patients for pharmacies with the median innovation-values score (13.07) was 32.09 compared to 59.36 for pharmacies with an implementation climate score at the 75th percentile (14.68).

Hypothesis 3

The marginal effect of innovation-values fit on the number of CMRs per high-risk patients increased as implementation climate score increased (Fig. 3).

Fig. 3
figure 3

Plot of marginal effect of innovation-values fit and implementation climate score for Equation 2

Full size image

Hypothesis 4

In terms of patients’ needs and resources, pharmacies located in rural locations were associated with lower implementation of CMRs per high-risk patients (p = 0.006). Conversely, pharmacies that participate in the 340B Drug Pricing Program were associated with higher implementation (p = 0.026). For available resources, pharmacies with a clinical pharmacist were associated with higher implementation (p = 0.002). However, an increase in total staff was associated with a 1.98 decrease in implementation of CMRs per high-risk patients (p < 0.001). For available resources, implementation of CMRs per high-risk patients was positively associated with experience with NC-CPESN (p = 0.004), past performance with NC-CPESN (p < 0.001), and participation in Medicare Part D MTM (p = 0.003). No significant differences in implementation were found based on structural characteristics.

Discussion

In this study, we used the organizational theory of innovation implementation effectiveness [18,19,20] to test organizational factors that influence implementation effectiveness of a community pharmacy medication management intervention. Consistent with our hypothesis, we found that key constructs from this theory, such as implementation climate and innovation-values fit, were positively associated with implementation and program reach of NC-CPESN. To our knowledge, only one other quantitative study has examined the relationship between implementation climate and implementation effectiveness in healthcare [31], and no other study in healthcare has explored the direct and indirect effects of innovation-values fit on implementation effectiveness. Contrary to our hypotheses of contextual factors, only certain factors, such as having a clinical pharmacist on staff, participation in Medicare Part D MTM, or 340B Drug Pricing Program, predicted both implementation and program reach. We were also surprised that 40.8% of the community pharmacies participating in the study did not have implementation activity within the study period. We describe potential reasons for this below.

We hypothesized that implementation climate and innovation-values fit would be positively and directly associated with implementation effectiveness, which was supported by our findings. These findings suggest that implementation climate and innovation-values fit were useful measures for predicting implementation and program reach. Further studies are needed to test whether these measures are predictive of implementation effectiveness across a wider variety of community pharmacy medication management programs. For example, we learned from the previous qualitative work that NC-CPESN community pharmacy staff worked collaboratively to implement CMRs. For other medication management programs, organizations may rely on a single staff member to deliver the intervention. In such cases, individual-referenced measures of implementation climate [32] may be more valid than group-referenced items.

The study results also supported the hypothesis that innovation-values fit moderates the effect of implementation climate on implementation effectiveness, indicating that implementation climate and innovation-values fit work in concert. Our findings also suggest that innovation-values fit may have a greater effect on implementation climate at lower levels of implementation climate and that the effect may diminish at higher levels of implementation climate (Fig. 2). Further research is needed to establish whether there are diminishing returns to the effect of innovation-values fit on implementation climate and whether the relationship depends on the outcome of interest, i.e., presence of implementation activity versus level of implementation activity (e.g., program reach). Additionally, future research is needed to determine what factors are positively associated with implementation climate and innovation-values fit in pharmacy medication management programs. For example, the organizational theory of innovation implementation effectiveness [18,19,20] maintains that management support is an antecedent of implementation climate, but there has been little quantitative research on how to operationalize the construct of management support. Recently, researchers have developed a measure for implementation leadership to assess which leadership qualities are correlated with successful implementation [44]. Future studies could assess whether implementation leadership is associated with implementation climate. This has practical importance because identifying the leadership behaviors and traits associated with effective implementation could provide guidance to pharmacy leaders on how to develop a supportive climate for medication management program implementation.

Contrary to our hypotheses, we found that only certain aspects of the organizational context affected both implementation and program reach. For example, none of the structural characteristics (e.g., pharmacy type, established pharmacy) were significantly associated with both implementation and program reach. Additional research is needed to determine whether there are other structural characteristics that may be associated with successful implementation of pharmacy medication management programs. Consistent with our hypotheses, access to knowledge about the intervention (e.g. participation in Medicare Part D MTM), patient needs and resources (e.g., proportion of high-risk patients, participation in 340B Drug Pricing Program), and availability of certain resources (e.g., clinical pharmacist) positively affect implementation effectiveness. Prior theory suggests that establishing an implementation climate for one intervention may help facilitate implementation climate for a similar intervention [29]. It is possible that community pharmacies use similar strategies to support MTM and medication management services implementation (e.g., staff training on motivational interviewing)—explaining the positive association between Medicare Part D MTM and NC-CPESN implementation. Future studies should use qualitative methods to explore the implementation strategies that community pharmacies establish to foster a climate for medication management services and whether these strategies facilitate implementation of similar interventions. Such studies could be used to develop implementation guidance to support community pharmacies participating in multiple medication management programs simultaneously, which may increase as pharmacy participation in alternative payment models grows. We also found that having a clinical pharmacist on staff was an important predictor of implementation effectiveness. However, clinical pharmacists’ roles and the amount of time available for clinical services can vary widely across community pharmacies [34, 35]. Future qualitative studies are needed to describe how community pharmacies define the job roles of clinical pharmacists, how much time clinical pharmacists are given for non-dispensing activities, and whether such differences are perceived to impact the effectiveness of clinical pharmacists.

Unexpectedly, we found that about 40% of the pharmacies participating in this study did not implement a CMR during the study period. From a recent qualitative study we conducted, we found that many of the pharmacies in the NC-CPESN program struggled not necessarily with conducting a CMR but with documenting a CMR [41]. Most pharmacies indicated that they received sufficient training on the documentation system but struggled with finding the time to document the CMR and found the process to be burdensome. Some pharmacies described having to hire additional staff to assist with documentation or having staff work overtime to keep up with documentation. To address this need, CCNC has updated the templates for CMR documentation and made improvements to the documentation system itself [45]. Future studies are needed to test whether the new system reduces the amount of time needed for documentation and improves NC-CPESN pharmacies ability to document CMRs. Such research would have applicability not only for NC-CPESN but also for other programs that require documentation of medication management services, such as the Medicare Part D MTM program.

Limitations

This study had several limitations. First, since we measured implementation climate, innovation-values fit, and implementation effectiveness at the same time, we cannot establish the causal order. Second, the generalizability of our findings is limited by: (1) only having data at one time point; (2) conducting the study in NC, the first CPESN organization. Future studies are needed to examine implementation of medication management programs over time and across settings. Third, our measures of implementation effectiveness, implementation and program reach among high-risk patients, are limited in scope and do not assess other important aspects of implementation effectiveness such as fidelity of CMR delivery. Future studies are needed to establish additional measures of implementation effectiveness (e.g., conducting site observations to measure CMR fidelity). Finally, we did not measure other determinants of implementation effectiveness including the presence of an innovation champion or variability in implementation climate perceptions [18,19,20]. Future studies should develop and test these measures in pharmacy medication management programs.

Conclusions

As more state Medicaid programs adopt pharmacist-led medication management programs, it is important to identify what organizational determinants promote effective implementation of these programs. Our study supported the use of the organizational theory of innovation implementation effectiveness to identify organizational determinants that are associated with effective implementation (e.g., implementation climate and innovation-values fit) [18,19,20]. Unlike broader environmental factors or structural characteristics (e.g., pharmacy type), implementation climate and innovation-values fit are modifiable factors and can be targeted through intervention—a finding that is important for community pharmacy practice. Additional research is needed to determine what implementation strategies can be used by community pharmacy leaders and practitioners to develop a positive implementation climate and innovation-values fit for medication management programs.

Abbreviations

AME:

Average marginal effect

CCNC:

Community Care of North Carolina

CMR:

Comprehensive medication review

CPESNSM :

Community Pharmacy Enhanced Services Network program

KMO:

Kaiser-Meyer-Olkin

MTM:

Medication therapy management

References

  1. CMS. Medicaid and CHIP: November 2014 monthly applications, eligibility determinants and enrollment report. 2015. https://www.medicaid.gov/medicaid/program-information/downloads/medicaid-and-chip-november-2014-application-eligibility-and-enrollment-report.pdf. Accessed 1 Jan 2018.

  2. Patient Protection and Affordable Care Act, 42 U.S.C. § 1396a(e)(14)(I) et seq. 2010.

  3. GAO. A small share of enrollees consistently accounted for a large share of expenditures (GAO-15-460). 2015. https://www.gao.gov/products/GAO-15-460. Accessed 1 Jan 2018.

  4. Neyarapally GA, Smith MA. Variability in state Medicaid medication management initiatives. Res Social Adm Pharm. 2017;13:214–23.

    Article  PubMed  Google Scholar 

  5. Urick B, Brown P, Easter JC. Achieving better quality and lower costs in Medicaid through enhanced pharmacy services. N C Med J. 2017;78:188–9.

    PubMed  Google Scholar 

  6. CCNC. Community Pharmacy Enhanced Services Network. 2016. https://www.communitycarenc.org/what-we-do/supporting-primary-care/pharmacy/cpesn. Accessed 21 Oct 2016.

  7. Chisholm-Burns MA, Kim Lee J, Spivey CA, Slack M, Herrier RN, Hall-Lipsy E, Graff Zivin J, Abraham I, Palmer J, Martin JR, et al. US pharmacists’ effect as team members on patient care: systematic review and meta-analyses. Med Care. 2010;48:923–33.

    Article  PubMed  Google Scholar 

  8. Beney J, Bero LA, Bond C. Expanding the roles of outpatient pharmacists: effects on health services utilisation, costs, and patient outcomes. Cochrane Database Syst Rev. 2000;3:CD000336.

  9. Blenkinsopp A, Anderson C, Armstrong M. Systematic review of the effectiveness of community pharmacy-based interventions to reduce risk behaviours and risk factors for coronary heart disease. J Public Health Med. 2003;25:144–53.

    Article  PubMed  Google Scholar 

  10. Kaboli PJ, Hoth AB, McClimon BJ, Schnipper JL. Clinical pharmacists and inpatient medical care: a systematic review. Arch Intern Med. 2006;166:955–64.

    Article  PubMed  Google Scholar 

  11. Viswanathan M, Kahwati LC, Golin CE, Blalock SJ, Coker-Schwimmer E, Posey R, Lohr KN. Medication therapy management interventions in outpatient settings: a systematic review and meta-analysis. JAMA Intern Med. 2015;175:76–87.

    Article  PubMed  Google Scholar 

  12. Doucette WR, Koch YD. An exploratory study of community pharmacy practice change. J Am Pharm Assoc (Wash). 2000;40:384–91.

    Article  CAS  Google Scholar 

  13. Roberts AS, Benrimoj SI, Chen TF, Williams KA, Aslani P. Practice change in community pharmacy: quantification of facilitators. Ann Pharmacother. 2008;42:861–8.

    Article  PubMed  Google Scholar 

  14. Feletto E, Wilson LK, Roberts AS, Benrimoj SI. Measuring organizational flexibility in community pharmacy: building the capacity to implement cognitive pharmaceutical services. Res Social Adm Pharm. 2011;7:27–38.

    Article  PubMed  Google Scholar 

  15. Pestka DL, Frail CK, Palombi LC, Von Hoff BA, Conway JM, Sorensen TD. Strategies and steps fostering the success of medication management services in community pharmacies. J Am Pharm Assoc (2003). 2016;56:504–12. e501

    Article  Google Scholar 

  16. Law AV, Okamoto MP, Brock K. Ready, willing, and able to provide MTM services?: a survey of community pharmacists in the USA. Res Social Adm Pharm. 2009;5:376–81.

    Article  PubMed  Google Scholar 

  17. Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. 2015;10:53.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Klein KJ, Sorra JS. The challenge of innovation implementation. Acad Manag Rev. 1996;21:1055–80.

    Article  Google Scholar 

  19. Klein KJ, Conn AB, Sorra JS. Implementing computerized technology: an organizational analysis. J Appl Psychol. 2001;86:811–24.

    Article  PubMed  CAS  Google Scholar 

  20. Helfrich CD, Weiner BJ, McKinney MM, Minasian L. Determinants of implementation effectiveness: adapting a framework for complex innovations. Med Care Res Rev. 2007;64:279–303.

    Article  PubMed  Google Scholar 

  21. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Crosby RA, Wendel ML, Vanderpool RC, Casey BR. Rural populations and health: determinants, disparities, and solutions. 1st ed. Hoboken: Wiley; 2012.

    Google Scholar 

  23. Freeman J, Carroll GR, Hannan MT. The liability of newness: age dependence in organizational death rates. Am Sociol Rev. 1983;48:692–710.

    Article  Google Scholar 

  24. Dalton DR, Todor WD, Spendolini MJ, Fielding GJ, Porter LW. Organization structure and performance: a critical review. Acad Manag Rev. 1980;5:49–64.

    Article  Google Scholar 

  25. CPESN-USA. What is CPESN? 2017. https://www.cpesn.com. Accessed 11 Sept 2017.

  26. CCNC. Community Care of North Carolina: A history. 2017. https://www.communitycarenc.org/knowledge-center/history-of-ccnc. Accessed 11 Sept 2017.

  27. Farley JF, Ferreri SP, Easter JC, McClurg MR. The North Carolina experiment: active research in the development and assessment of new practice models. N C Med J. 2017;78:186–90.

    PubMed  Google Scholar 

  28. Smith M, Ferreri S, Stafford R, Teeter B, Turner K, Veach S, Shea CM. Development of community pharmacy characteristics questionnaire: application of cognitive interviewing. Innovations Pharm. 2018; In press

  29. Weiner BJ, Belden CM, Bergmire DM, Johnston M. The meaning and measurement of implementation climate. Implement Sci. 2011;6:78.

    Article  PubMed  PubMed Central  Google Scholar 

  30. RWJF. North Carolina county rankings. 2017. http://www.countyhealthrankings.org/app/north-carolina/2017/overview. Accessed 1 Jan 2018.

  31. Jacobs SR, Weiner BJ, Reeve BB, Hofmann DA, Christian M, Weinberger M. Determining the predictors of innovation implementation in healthcare: a quantitative analysis of implementation effectiveness. BMC Health Serv Res. 2015;15:6.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Jacobs SR, Weiner BJ, Bunger AC. Context matters: measuring implementation climate among individuals and groups. Implement Sci. 2014;9:46.

    Article  PubMed  PubMed Central  Google Scholar 

  33. HRSA. Contract pharmacy services. 2018. https://www.hrsa.gov/opa/implementation/contract/index.html. Accessed 26 July 2018.

  34. AACP. Standards of practice for clinical pharmacists. 2018. https://www.accp.com/docs/positions/guidelines/standardsofpractice.pdf. Accessed 5 July 2018.

  35. McDonough RP. Community pharmacists: retailers or clinicians? Pharm Times. 2016;22:49.

    Google Scholar 

  36. Cameron AC, Trivedi PK. Regression analysis of count data. Cambridge: Cambridge University Press; 2013.

    Book  Google Scholar 

  37. Cragg JG. Some statistical models for limited dependent variables with application to the demand for durable goods. Econometrica. 1971;39:829–44.

    Article  Google Scholar 

  38. Karaca-Mandic P, Norton EC, Dowd B. Interaction terms in nonlinear models. Health Serv Res. 2012;47:255–74.

    Article  PubMed  Google Scholar 

  39. Keele L, Kelly NJ. Dynamic models for dynamic theories: the ins and outs of lagged dependent variables. Polit Anal. 2005;14:186–205.

    Article  Google Scholar 

  40. Harris RDF, Tzavalis E. Inference for unit roots in dynamic panels where the time dimension is fixed. J Econ. 1999;91:201–26.

    Article  Google Scholar 

  41. Turner K, Renfro C, Ferreri S, Roberts K, Pfeiffenberger T, Shea CM. Supporting community pharmacies with implementation of a web-based medication management application. Appl Clin Inform. 2018;9:391–402.

    Article  PubMed  Google Scholar 

  42. Kaiser HF. An index of factorial simplicity. Psychometrika. 1974;39:31–6.

    Article  Google Scholar 

  43. DeVellis RF. Scale development: theory and applications. Thousand Oaks: Sage Publications; 2016.

  44. Aarons GA, Ehrhart MG, Farahnak LR. The implementation leadership scale (ILS): development of a brief measure of unit level implementation leadership. Implement Sci. 2014;9:45.

    Article  PubMed  PubMed Central  Google Scholar 

  45. CPESN. Pharmacist eCare plan gains momentum. 2017. https://cpesn.com/newsroom/pharmacist-ecare-plan-gains-momentum/. Accessed 21 Dec 2017.

Download references

Acknowledgements

The authors would like to thank the Community Care of North Carolina staff for their assistance with survey development and administration and for sharing their expertise about the Community Pharmacy Enhanced Services Network program. The authors would especially like to thank Troy Trygstad for his support and assistance.

Funding

This study was funded by grants from the Community Pharmacy Foundation (71560) and the North Carolina Translational and Clinical Sciences Institute (UL1TR001111). Additionally, the project described in this study was supported by funding opportunity number 1C12013003897 from the U.S Department of Health and Human Services, Centers for Medicare and Medicaid Services. The contents provided are solely the responsibility of the authors and do not necessarily represent the official views of HHS or any of its agencies or other funders of this study.

Availability of data and materials

To protect participants’ confidentiality, individual-level data cannot be provided. Aggregated data can be made available on request.

Author information

Authors and Affiliations

  1. Department of Health Policy and Management, Gillings School of Global Public Health, The University of North Carolina at Chapel Hill, 135 Dauer Drive, Chapel Hill, NC, 27599-7411, USA

    Kea Turner, Justin G. Trogdon, Morris Weinberger & Angela M. Stover

  2. Division of Practice Advancement and Clinical Education, Eshelman School of Pharmacy, The University of North Carolina at Chapel Hill, 115B Beard Hall, Chapel Hill, NC, 27599-7411, USA

    Stefanie Ferreri & Michael Patti

  3. Department of Pharmaceutical Care and Health Systems, College of Pharmacy, University of Minnesota, 308 Harvard Street SE, Minneapolis, MN, 55455, USA

    Joel F. Farley

  4. Center for Medication Optimization through Practice and Policy, Eshelman School of Pharmacy, The University of North Carolina at Chapel Hill, 2400 Kerr Hall, Chapel Hill, NC, 27599-7411, USA

    Neepa Ray

  5. Department of Clinical Pharmacy and Translational Science, University of Tennessee Health Science Center, 881 Madison Avenue, Memphis, TN, 38163, USA

    Chelsea Renfro

  6. Department of Health Policy and Management, Gillings School of Global Public Health, The University of North Carolina at Chapel Hill, 1103E McGavran-Greenberg, 135 Dauer Drive, Chapel Hill, NC, 27599-7411, USA

    Christopher M. Shea

Authors
  1. Kea Turner
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Justin G. Trogdon
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Morris Weinberger
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Angela M. Stover
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Stefanie Ferreri
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Joel F. Farley
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Neepa Ray
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Michael Patti
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Chelsea Renfro
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Christopher M. Shea
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

KT led the research design, data collection and analysis, and writing of the manuscript. JT, MW, and AS assisted with the data analysis. SF, JF, NR, MP, and CR assisted with the research design and data collection processes. CS assisted with the research design and data collection and analysis processes. All authors reviewed the manuscript, provided substantial feedback, and approved the final draft of the manuscript.

Corresponding author

Correspondence to Kea Turner.

Ethics declarations

Ethics approval and consent to participate

Participants provided written consent to participate in this study. The Institutional Review Board of the University of North Carolina at Chapel Hill approved this study (IRB # 17-1304).

Consent for publication

Participants also provided written consent for the publication of study findings.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Turner, K., Trogdon, J.G., Weinberger, M. et al. Testing the organizational theory of innovation implementation effectiveness in a community pharmacy medication management program: a hurdle regression analysis. Implementation Sci 13, 105 (2018). https://doi.org/10.1186/s13012-018-0799-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13012-018-0799-5

Keywords

Implementation Science

ISSN: 1748-5908