- Open Access
- Open Peer Review
Audience segmentation to disseminate behavioral health evidence to legislators: an empirical clustering analysis
Implementation Science volume 13, Article number: 121 (2018)
Elected officials (e.g., legislators) are an important but understudied population in dissemination research. Audience segmentation is essential in developing dissemination strategies that are tailored for legislators with different characteristics, but sophisticated audience segmentation analyses have not been conducted with this population. An empirical clustering audience segmentation study was conducted to (1) identify behavioral health (i.e., mental health and substance abuse) audience segments among US state legislators, (2) identify legislator characteristics that are predictive of segment membership, and (3) determine whether segment membership is predictive of support for state behavioral health parity laws.
Latent class analysis (LCA) was used. Data were from a multi-modal (post-mail, e-mail, telephone) survey of state legislators fielded in 2017 (N = 475). Nine variables were included in the LCA (e.g., perceptions of behavioral health treatment effectiveness, mental illness stigma). Binary logistic regression tested associations between legislator characteristics (e.g., political party, gender, ideology) and segment membership. Multi-level logistic regression assessed the predictive validity of segment membership on support for parity laws. A name was developed for each segment that captured its most salient features.
Three audience segments were identified. Budget-oriented skeptics with stigma (47% of legislators) had the least faith in behavioral health treatment effectiveness, had the most mental illness stigma, and were most influenced by budget impact. This segment was predominantly male, Republican, and ideologically conservative. Action-oriented supporters (24%) were most likely to have introduced a behavioral health bill, most likely to identify behavioral health issues as policy priorities, and most influenced by research evidence. This was the most politically and ideologically diverse segment. Passive supporters (29%) had the greatest faith in treatment effectiveness and the least stigma, but were also least likely to have introduced a behavioral health bill. Segment membership was a stronger predictor of support for parity laws than almost all other legislator characteristics.
State legislators are a heterogeneous audience when it comes to behavioral health. There is a need to develop and test behavioral health evidence dissemination strategies that are tailored for legislators in different audience segments. Empirical clustering approaches to audience segmentation are a potentially valuable tool for dissemination science.
When informed by research evidence, legislators’ decisions can radically accelerate the population impact of mental health and substance abuse research (hereafter referred to as behavioral health) [1,2,3,4,5,6]. For example, evidence about the benefits of insurance regulations (e.g., state parity laws) [7, 8], effective treatments, preventive interventions, and macro determinants of behavioral health (e.g., structural stigma) [9, 10] can be translated into action through laws that the 7383 state legislators in the USA have the exclusive authority to enact . For these reasons, the importance of disseminating behavioral health evidence to legislators has been articulated by actors ranging from behavioral health professionals  to the US National Academies [13, 14]. Although the rationale for disseminating behavioral health evidence to legislators is clear, there is sparse empirical guidance about how to effectively do it [6, 15,16,17].
Policy dissemination research—defined as the study of the targeted distribution of evidence to policymakers —has almost exclusively focused on physical health [16, 19, 20]. A 2015 review of interventions to increase the use of mental health research in policymaking found few studies, and none focused on legislators . A review of policy dissemination research funded by the National Institutes of Health between 2007 and 2014 did not identify any projects focused on disseminating behavioral health evidence to legislators or other policymakers in the USA . Effective policymaker-focused dissemination strategies are presumably different for behavioral health than physical health because stigma towards people with behavioral health conditions is pervasive [13, 21,22,23,24,25,26] and because willingness to allocate financial resources is lower for behavioral health than physical health services [27, 28]. Furthermore, while policy dissemination research is a rapidly growing field outside of the USA , these studies have primarily focused on administrative officials who have specialized knowledge about health. Elected policymakers, such as legislators, generally lack such knowledge and thus have distinct dissemination needs . In the absence of empirical evidence about how behavioral health research might be most effectively disseminated to elected policymakers, dissemination activities are typically based on conjecture about what will work and an assumption about how information might be perceived by policymakers with different characteristics.
As Leeman and colleagues note in their typology of implementation strategies, audience segmentation research is an essential first step towards understanding how to effectively disseminate evidence to legislators or any target population . Audience segmentation theory is founded on the premise that a population’s members are heterogeneous in their knowledge and attitudes and that tailored dissemination strategies that reflect these differences are more effective and persuasive than “one-size-fits-all” dissemination . Routine in the fields of marketing and communication, the practice of audience segmentation entails formative assessment to identify discrete sub-populations (i.e., audience segments) that are similar in their attitudes and behaviors . Messages that are tailored to audience segments are generally more effective than non-tailored messages [32, 33]. Despite the importance of audience segmentation, policymakers have typically been treated as a homogenous population in policy dissemination research in the USA  and sophisticated audience segmentation analyses have not been conducted.
There are two main approaches to audience segmentation: demographic separation and empirical clustering . With demographic separation, a population is divided into audiences on the basis of demographic characteristics (e.g., gender, political party affiliation) with the assumption that there is homogeneity within demographic groups. A more sophisticated approach, empirical clustering, uses statistical techniques (e.g., latent class analysis, k-means clustering) to identify audience segments based on relationships among relevant variables. Empirical clustering is considered the superior approach  and has been used to identify audience segments among the general public related to issues such as climate change [35,36,37,38,39] and health equity . To our knowledge, no studies have used an empirical clustering approach to identify audience segments related to behavioral health among the general public or policymakers. More broadly, empirical clustering approaches to audience segmentation have not been widely used in the field of dissemination science.
Study aims and purpose
We conducted an empirical clustering audience segmentation study using latent class analysis (LCA) of data from a survey of US state legislators. The aims were to:
Identify behavioral health audience segments among state legislators
Identify legislator characteristics that are predictive of segment membership
Determine whether segment membership is predictive of support for state behavioral health parity laws
The purpose of the study was to inform the design of dissemination strategies that are tailored for legislators in different audience segments, with ultimate goal of improving evidence-informed behavioral health policymaking among state legislators. It should be emphasized, however, that the use of research evidence in policymaking is extremely complex and that there are myriad barriers to evidence-informed policymaking (e.g., lack of trust in researchers, lack of policy relevant research findings, limited capacity to use research) [19, 41,42,43,44]. Audience segmentation, or improved dissemination more broadly, is not a solution to these issues [45, 46]. Rather, it simply provides an empirical basis for packaging and communicating research evidence in ways that more accurately reflect policymakers’ characteristics.
Study design and data
Between March and September 2017, we conducted a multi-modal (post-mail, e-mail, telephone) survey of US state legislators (study protocol published in Implementation Science) . To construct the sample frame, we used the National Conference of State Legislatures’ (NCSL) database and randomly selected up to 60 legislators from every state who were legislators as of January 15, 2017. We excluded legislators who had just entered office on January 1, 2017, because they would have limited legislative experience at the time of survey completion. After excluding legislators whose contact information was invalid and those who left office prior to when data collection began, the sample frame consisted of 2902 legislators.
Each legislator received a post-mailed invitation to complete the survey online, two post-mailed paper versions of the survey with self-addressed postage-paid return envelopes, ten e-mail invitations to complete the survey online, and up to 15 telephone calls. In total, a legislator who did not complete the survey was contacted 29 times. Recruitment materials stated that only the legislator, not their staff, could complete the survey. Data were collected by a survey research firm, and Institutional Review Board approval was obtained.
The survey was completed by 475 legislators, and the response rate was 16.4%, which is reasonable for state legislators  and is higher than response rates of other recent state legislator surveys [49,50,51]. Respondents were significantly more likely than non-respondents to be Democrat (48.8% vs. 42.4%, χ2 = 10.19, p = .001), female (32.6% vs. 23.0%, χ2 = 19.73, p < .001), and from the Midwest US Census Region (30.5% vs. 22.5%, χ2 = 14.27, p < .001). To account for these differences and make our results more generalizable to the entire population of state legislators, we calculated and applied non-response weights accounting for political party, gender, and geographic region using a sample post-stratification approach in which weighting classes were based on the full sampling frame . Weighted and unweighted results were only modestly different, suggesting that results were not strongly driven by legislator characteristics specific to the sample.
Variables and analysis
The survey instrument (Additional file 1) contained 59 items and was vetted by NCSL and cognitively pre-tested with five former state legislators prior to fielding. Most survey items were adapted from questions previously used in public opinion surveys about behavioral health issues or legislator surveys about research use in policymaking .
Consistent with recommendations for cognitive pre-testing , the draft survey was e-mailed to former state legislators and telephone-based interviews were conducted to assess whether questions and response options were clear. The main finding from these interviews was that mental health and substance use disorders were through of separate, although often related, issues. Thus, the survey was revised to explicitly anchor questions to “mental health” and/or “substance use disorder” issues as opposed to only using language of “mental health” or “behavioral health.”
Nine variables that spanned five domains were included in the LCA. All variables were dichotomous to aide interpretability. Variable selection was informed by Watson and Corrigan’s framework of factors that influence policymakers’ behavioral health policy decisions  and McGinty and colleagues’ review of communication strategies to reduce stigma and generate support for behavioral health policies . We did not seek to map variables onto the framework or review, but rather used these sources to orient the selection of variables that are likely to be associated with legislators’ behavioral health policymaking behaviors.
Perceptions of behavioral health treatment effectiveness (two variables)
Legislators separately rated the extent to which they agreed that “mental health treatments can help people with mental illness lead normal lives” and “substance disorder treatments can help people with a substance use disorder recover” (1 = “strongly disagree,” 5 = “strongly agree”). The mental health treatment effectiveness question has been previously used in Behavioral Risk Factor Surveillance System surveys  and was adapted by the authors to assess perceptions of substance use disorder treatment effectives. These variables were dichotomized as “strongly agree,” yes/no.
Stigma towards people with mental illness (one variable)
Stigma was assessed using a composite measure based on four items used by Barry and McGinty  to assess stigma among the US general public. Two items assessed attitudes about the dangerousness of people with mental illness  and two assessed preferences for social distance from people with mental illness . Cronbach’s alpha for the four items was .80, similar to when the four items were used with the general public (Cronbach’s alpha = .77) . We coded responses so that higher scores corresponded with greater stigma and summed them to create a stigma score for each respondent (range = 0 to 16). In the absence of a priori theoretically or empirically relevant cut-points, we classified legislators based on sample quartiles of the score.
Factors that have the most influence on support for a behavioral health bill (two variables)
Legislators selected up to two factors, from a list of five options, that “have the most influence on whether [they] support” a “mental health/substance abuse bill” when it is introduced. The list of factors was developed by the authors, informed by literature about decision making among state legislators . We classified each legislator according to whether or not they selected “the extent to which the bill is going to impact the state budget” and “is based on scientific evidence.”
Most important health issues for legislative action in the state (two variables)
Legislators were presented with a list of 19 health issues and selected up to three that were most important “for legislative action in [their] state.” This question and the list of health issues have been previously used in surveys of state legislators [58, 59]. We classified each legislator according to whether they selected “mental health” or “substance abuse,” respectively. We also reviewed open-ended response options and coded responses accordingly (e.g., coded “access the mental health services” as mental health and coded “opioid epidemic” as substance abuse).
History of introducing a behavioral health bill (two variables)
Legislators separately indicated whether they had ever introduced a “mental health” or “substance abuse” bill.
Identification of audience segments
Latent class analysis (LCA) is considered the optimal analytic strategy for audience segmentation . We used the SAS PROC LCA add-on  to identify audience segments of legislators based on the clustering of their responses to the nine variables that were selected for inclusion in the LCA . We fit models using three through five segment solutions and then selected the most appropriate model based on the criteria of interpretability, identification, and fit statistics. Fit statistics included the adjusted Aikake Information Criterion, Bayesian Information Criterion (BIC), and sample-size-adjusted BIC . To assess model identification, we estimated the parameters for each model using 1000 random starting values and measured the proportion of iterations that converged to the same maximum likelihood solution . Based on these criteria, we found that a three segment solution was optimal (Table 1). We then developed a name for each segment that concisely captured its most salient and distinct features .
Identification of predictors of segment membership
In order to test predictors of segment membership, we assigned each legislator to the segment for which they had the highest posterior probability of membership. We then used binary logistic regression to test associations of legislator characteristics with membership in the three segments. We modeled membership in each class segment as a separate model, using the other two segments combined as the referent group.
For each segment, we used a sequential modeling approach in which we first included legislator characteristics for which information is readily and publicly available (i.e., political party, gender, current health committee membership, number of years serving as a legislator, and US Census Region). Then, in a second model, we added characteristics that would typically require additional primary data collection (i.e., social and fiscal ideology, highest level of education). Social and fiscal ideologies were assessed separately in the survey using items adapted from American National Election Studies’ questionnaires  that have been previously used with state legislators .
We used odds ratios and 95% confidence intervals (CIs) to quantify associations of each potential predictor with class membership after adjustment for the other variables in the model. Because our method of assigning participants to discrete segments did not account for uncertainty in segment membership, we conducted a sensitivity analysis in which the legislator characteristic variables were included as covariates in the estimation of the LCA . Results were consistent, and we present the results using separate logistic regression models for ease of interpretability.
Determining the predictive validity of segment membership
To assess the predictive validity of segment membership on behavioral health policy support, we used multi-level logistic regression to examine associations between segment membership and support for state behavioral health parity laws. The multi-level models used state-level random intercepts to account for correlated responses of legislators from the same state. We focused on parity laws because they are an evidence-based policy recommended by the US Task Force on Community Preventive Services [7, 8]. A definition of behavioral health parity laws was provided (see Additional file 1), and legislators rated the extent to which they supported them (1 = “strongly oppose,” 5 = “strongly support”). This variable was dichotomized as “strongly support,” yes/no. We ran unadjusted models as well as models adjusted for legislator characteristics. The adjusted models allowed us to determine whether segment membership was predictive of policy support independent of these covariates and compare predictive power of segment membership relative to non-latent variables.
Characteristics of audience segments and predictors of segment membership
Table 2 presents data on the distribution of the nine LCA variables across the three segments, Table 3 shows the demographic characteristics of legislators in each segment, and Table 4 presents logistic regression results showing adjusted associations between each demographic characteristic and segment membership. The narrative below describes the characteristics of each of the audience segments.
Budget-oriented skeptics with stigma
This constituted the largest segment of legislators (47%). Legislators in this segment were the most skeptical about the potential effectiveness of mental health and substance abuse treatments—with only 16.9% and 12.6%, respectively, strongly agreeing that treatments can help people lead normal lives and recover (Table 2). Legislators in this segment also had much more stigma towards people with mental illness than legislators in the other segments, with 34.2% having stigma scores in the fourth quartile (most stigma) and 42.6% in the third quartile. Among legislators in this segment, more reported that the impact that a behavioral health bill will have on the state budget (61.4%) influenced their support than the extent to which the bill is based on evidence (46.1%).
Unlike legislators in the other two segments, the majority of legislators in this segment were Republican (73.6%), socially conservative (66.1%), and fiscally conservative (78.3%) (Table 3). However, political party was not significantly associated with segment membership after adjustment for all other predictors (Table 4, model 2). This was also the most predominantly male segment (83.8%). After adjustment, the odds of a male legislator being in this segment was nearly twice that of a female (Table 4; aOR = 1.72, 95% CI = 1.03, 2.87).
This was the smallest segment of legislators (24%). Legislators in this segment were, by far, the most likely to have introduced a behavioral health bill, with 90.7% having introduced a mental health bill and 96.3% having introduced a substance abuse bill (Table 2). Legislators in this segment were also the most likely to identify substance abuse issues as policy priorities (58.3%) and nearly half (45.6%) also prioritized mental health issues. These legislators were more likely to report that the extent to which a bill was based on evidence (74.1%) influenced whether they supported a behavioral health bill than the impact it would have on the state budget (29.2%).
Unlike legislators in the other two segments, the majority of legislators in this segment were on a health committee (60.8%) (Table 3). After adjustment, health committee members had nearly four times the odds of being in this segment as legislators who were not health committee members (aOR = 3.94, 95% CI = 2.44, 6.37) (Table 4, model 2). Legislators in this segment also were more likely to have been long-time legislators, with 70.9% having been a legislator for ≥ 6 years compare to less than half of the legislators in the other two segments (Table 3). After adjustment, the odds of a legislator with ≥ 6 years of experience being in this segment was nearly three times as that of a legislator with < 6 years of experience (aOR = 2.78, 95% CI = 1.70, 4.53) (Table 4, model 2). This was also the most ideologically diverse segment, with the modal political party being democrat (51.5%), the modal social ideology being moderate (41.9%), and the modal fiscal ideology being conservative (51.8%) (Table 3).
Twenty-nine percent of legislators belonged to this segment. Legislators in this segment had the most faith in treatment effectiveness with 98.9% and 84.8%, respectively, strongly agreeing that treatments can help people lead normal lives and recovery (Table 2). These legislators also had the least stigma towards people with mental illness with only 7.8% having stigma scores in the fourth quartile (most stigma) and 18.5% in the third quartile. Despite their faith in the effectiveness of behavioral health treatments and low levels of stigma, a relatively small proportion had introduced a mental health bill (23.2%) and barely any (4.6%) had introduced a substance abuse bill.
This was the segment with the most gender diversity (34.2% female, 65.9% male; Table 3). It was also the segment with the largest proportion of Democrats (66.9%), and over one fifth identified as fiscally and socially liberal (27.6% and 20.4%, respectively). After adjustment, the odds of a socially liberal legislator being in this segment were nearly three times that of a socially conservative legislator (aOR = 2.94, 95% CI = 1.25, 6.90) (Table 4, model 2).
Predictive validity of segment membership on support for state parity laws
Table 5 shows unadjusted and adjusted associations of segment membership with strong support for behavioral health parity laws. The proportion of legislators strongly supporting state behavioral health parity laws was lowest in the budget-oriented skeptics with stigma segment (15.8%) and similar between the action-oriented proponent (66.4%) and passive believer segments (65.1%). After adjustment for legislator characteristics and state-level correlations, the odds of strong support for state parity laws were three times higher among passive supporters (aOR = 3.47, 95% CI = 1.83, 6.60) and more than six times higher among action-oriented supporters (aOR = 6.67, 95% CI = 3.30, 13.46) than budget-oriented skeptics with stigma (Table 5). Importantly, segment membership was a stronger predictor of strong support for state parity laws than almost all other socio-demographic characteristics.
Our results demonstrate that US state legislators are heterogeneous in their knowledge and attitudes about behavioral health and that at least three distinct audience segments exist: budget-oriented skeptics with stigma who have the least faith in behavioral health treatment effectiveness, have the most mental illness stigma, are most influenced by budget impact, and are ideologically conservative; action-oriented supporters who are most likely to have introduced a behavioral health bill, are most likely to identify behavioral health issues as policy priorities, are most influenced by research evidence, and are ideologically diverse; and passive supporters who have the greatest faith in behavioral health treatment effectiveness and the least mental illness stigma, but are also least likely to have introduced a behavioral health bill.
Membership in these latent audience segments had much higher predictive validity for support of state behavioral health parity laws than non-latent legislator characteristics (e.g., political party, education level). These findings underscore the importance of, and provide an empirical foundation for, developing and testing behavioral health dissemination strategies that are tailored to these different audience segments of legislators.
Implications for tailored dissemination strategies
Budget-oriented skeptics with stigma are a priority population to target given that they are the largest segment of legislators (47%) and have the highest levels of mental illness stigma and the least faith in behavioral health treatment effectiveness. As these are likely the political actors who contribute to structural stigma through policies that restrict opportunities for people with mental illness [9, 10], there is an urgent need to disseminate information that reduces stigma among these legislators. Effective communications interventions to reduce stigma among the general public exist [13, 22, 66], and research is needed to understand how these interventions might be tailored for legislators in this segment. These dissemination efforts should be tailored to the conservative characteristics of this segment (e.g., 73.6% Republican, 66.1% socially conservative). Dissemination might be most effective if information is framed in ways that resonate with a conservative worldview [67, 68] and if messages originate from sources that conservative legislators perceive as credible. Post hoc analyses (Additional file 2) revealed that legislative staff and state behavioral health agencies were the primary sources that legislators in this segment turned to for behavioral health research when making policy decisions. Thus, these sources might be important intermediaries to target in dissemination efforts.
Less than 20% of legislators in the budget-oriented skeptics with stigma strongly agreed that behavioral health treatments were effective, compared to over 70% of legislators in the other two segments. As Watson and Corrigan note, legislators’ perceptions of behavioral health treatment effectiveness often represent concerns about wasting finite resources . This is consistent with our findings that legislators in this segment were much more influenced by budget impact when making behavioral health policy decisions than any other group and were by far the most fiscally conservative (78.3%). Thus, messages targeting this segment might emphasize the costs of unaddressed behavioral health problems and the potential return on investment for prevention and treatment [14, 69,70,71]. However, it should be noted that socially conservative ideology, in addition to fiscally conservative ideology, was a strong predictor of segment membership in the fully adjusted model. This suggests that fiscal concerns are not the only core attribute of this segment. Attitudes towards people with mental illness (e.g., perceptions of the extent to which their problems are the result of individual versus structural issues) [5, 26, 72] and other characteristics often associated with people with mental illness (e.g., low social class, minority race/ethnicity)  potentially play an important role and should be considered in dissemination strategies.
Although action-oriented supporters was the smallest segment of legislators (24%), it is promising that this segment is characterized by prioritizing behavioral health issues, introducing behavioral health bills, and being strongly influenced by evidence when making behavioral policy decisions. It is also encouraging that this was the most politically and ideological diverse segment (e.g., 51.5% Democrat, 45.4% Republican). These findings are consistent with a 2012 survey of state legislators that found bipartisan support for behavioral health issues and that legislators who prioritized behavioral health issues were more influenced by research evidence than legislators’ who did not prioritize these issues . Taken together, these findings suggest that dissemination efforts targeting legislators in the action-oriented supporters segment should include concrete information about the science supporting evidence-based policy options to address behavioral health issues. Post hoc analyses (Additional file 2) showed that legislators in this segment identified behavioral health advocacy organizations as their primary source of behavioral health research. Thus, these organizations should be a target for the dissemination of this information. Passive supporters might be the lowest priority population to target given their exceptionally high faith in behavioral health treatment effectiveness and relatively low levels of mental illness stigma. Although legislators in this segment are least likely to have introduced behavioral health bills, they are similarly influenced by research evidence as action-oriented supporters when deciding whether to support behavioral health bill. Research should assess whether legislators in this segment respond similarly to dissemination strategies that are tailored for legislators in the action-oriented supporters segment.
Implications for knowledge translation
The current study focused on generating information to enhance the precision of dissemination efforts that push research evidence to legislators. Our results, however, also have potential implications for knowledge translation efforts more broadly . Specifically, there could be implications for efforts that facilitate the pull of research by legislators in the budget-oriented skeptics with stigma segment and exchange efforts that foster relationships between behavioral health researchers and legislators in this segment.
In terms of pull efforts, evidence clearinghouses could include economic evaluation data given that considerations related to budget impact were of high importance to legislators in this segment. The Washington State Institute for Public Policy’s Benefit-Costs Results clearinghouse is one model that could be adapted in states with conservative legislatures . In terms of implications for exchange efforts, interventions that foster positive relationships between behavioral health researchers and legislators in the budget-oriented skeptics with stigma category could increase trust in researchers and potentially increase research use. Post hoc analyses (Additional file 2) showed that only 18.3% of legislators in this segment identified universities as a primary source of behavioral health research, a proportion significantly lower than the other two segments. This is consistent with the results of a 2012 state legislator survey which found that social and fiscal conservative ideology—strong predictors of membership in the budget-oriented skeptics with stigma—was inversely associated with the extent to which universities were perceived as reliable sources of research . Trusted intermediary organizations such as the American Legislative Exchange Council—an ideologically conservative legislator assistance organization—could help broker these relationships. A knowledge exchange intervention that recently demonstrated success with federal legislators in the USA is a model that could be adapted .
Implications for dissemination science and future research
Our study demonstrates the utility of empirical clustering approaches to audience segmentation in policy dissemination research. Compared to prior studies that used demographic separation approaches to identify audience segments of legislators [58, 59, 65, 78], our empirical clustering approach produced a more nuanced understanding of how evidence about a specific issue (i.e., behavioral health) might be most effectively packaged for different types of legislators. Although our study was focused on legislators in the USA, dissemination studies targeting administrative (i.e., not elected) policymakers, such as those being conducted in Australia  and Canada , might consider how empirical clustering could be used to identify audience segments within government agencies.
The value of identifying audience segments hinges upon the extent to which dissemination strategies that are tailored for these segments are more effective than non-tailored strategies. Narratives (i.e., stories about people) are a medium that can be integrated into dissemination materials, tailored for different audience segments, and manipulated by researchers to test dissemination effects. Narratives are important in policymaking processes because they are engaging and evocative, can humanize abstract problems, and illustrate how contextual factors (that can often be modified by policies) affect individuals [81, 82]. Two experiments have tested the effects of narrative-focused dissemination materials on support for evidence-based policies among state legislators (one study focused on cancer , one focused on obesity ), and numerous studies have tested the effects of narratives about behavioral health issues on policy support among the general public [26, 83,84,85].
For example, a recent public opinion experiment by McGinty et al.  tested the effects of narratives that framed issues related to people with mental illness in different ways. The study found that narratives that emphasized systematic barriers to mental health treatment were more effective at increasing public willingness to pay additional taxes to improve the mental health system than a narrative about successful treatment-and-recovery. As McGinty et al. note (p. 212), future research should build on these studies and test the effects of such narratives on policymakers as opposed to the general public. By identifying behavioral health audience segments of state legislators, the current study could inform the tailoring and enhance the precision of policymaker-targeted narratives about behavioral health issues.
Measuring the effects of dissemination strategies on policymakers, particularly legislators, can be challenging [18, 86]. Proximal measures of the effectiveness of tailored versus non-tailored dissemination materials could include perceptions of the materials (e.g., perceived likelihood of using the information, clarity, and relevance) and support for evidence-based policies that are the focus of the dissemination materials. Such outcomes have been previously assessed among state legislators via brief surveys that accompany materials [50, 87, 88].
More distal measures of effectiveness could include legislators’ research use and policymaking behaviors. Information on these outcomes could be obtained via unobtrusive measures such as legislative voting [89,90,91], committee hearings [92, 93], the content of bills and other legislative documents , public statements , and social media behaviors . Structured interviews that assess the use of research evidence in policymaking, such as the Staff Assessment of enGagement with Evidence instrument [97, 98], could also be adapted for state legislators to examine differences between those who receive tailored versus non-tailored dissemination materials.
Our study has five main limitations. First, while our response rate of 16.4% is reasonable for state legislators  and higher than response rates of recent legislator surveys [49,50,51], it is sub-optimal by typical health services research standards. Demographic information about non-respondents allowed us to determinate that respondents were significantly different than non-respondents in terms of their political party affiliation, gender, and geographic region and to develop and apply non-response weights to adjust for these differences. This increases our confidence that results are not biased by non-response issues.
Second, our stigma measures were focused on mental illness and not able to assess stigma towards people with substance use disorders. This distinction could be important because evidence suggests that the public holds has more negative and stigmatizing attitudes towards people with substance use disorders than mental illness [22, 24].
Third, our survey questions were broadly focused on behavioral health issues and we did not explicitly anchor questions to adult or child populations. While some of our questions implied that we were asking about adults (e.g., willingness to work closely with someone who has serious mental illness), it is not clear whether respondents were thinking of adults or children, or both, when answering questions. Perceptions of adults’ behavioral health issues, and policy solutions to address them, are often different than perceptions of children’s mental health issues [22, 99,100,101,102], and different audience segments might exist for children’s behavioral health.
Fourth, our study was limited to elected policymakers in the legislative branch of government at the state-level in the USA and results are not necessarily generalizable to elected policymakers at different levels of government, those outside the USA, or administrative policymakers in executive branches of government.
Fifth, a limitation of LCA is the risk of misclassifying of individuals, particularly those whose posterior segment membership probabilities are far from 0 or 1. In our analysis, mean posterior probabilities for segment membership ranged from 0.91 to 0.94 and mean probabilities for segment non-membership ranged from 0.03 to 0.06, which is encouraging .
State legislators are a heterogeneous audience when it comes to behavioral health. There is a need to develop and test behavioral health evidence dissemination strategies that are tailored for legislators in different audience segments. Empirical clustering approaches to audience segmentation are a potentially valuable tool for dissemination science.
Raghavan R, Bright CL, Shadoin AL. Toward a policy ecology of implementation of evidence-based practices in public mental health settings. Implement Sci. 2008;3:26.
PEW Foundation: Mental Health and the Role of the States. Available from: http://www.pewtrusts.org/~/media/assets/2015/06/mentalhealthandroleofstatesreport.pdf. Accessed 10 Apr 2018.
Shonkoff JP, Bales SN. Science does not speak for itself: translating child development research for the public and its policymakers. Child Dev. 2011;82:17–32.
Insel TR. Translating scientific opportunity into public health impact: a strategic plan for research on mental illness. Arch Gen Psychiatry. 2009;66:128–33.
Corrigan PW, Watson AC. Factors that explain how policy makers distribute resources to mental health services. Psychiatr Serv. 2003;54:501–7.
Purtle J, Brownson RC, Proctor EK. Infusing science into politics and policy: The importance of legislators as an audience in mental health policy dissemination research. Adm Policy Ment Health Ment Health Serv Res. 2017;44(2):160–3.
Sipe TA, Finnie RK, Knopf JA, Qu S, Reynolds JA, Thota AB, Hahn RA, Goetzel RZ, Hennessy KD, McKnight-Eily LR. Effects of mental health benefits legislation: a community guide systematic review. Am J Prev Med. 2015;48:755–66.
Finding TF. Recommendation for mental health benefits legislation. Am J Prev Med. 2015;48:767–70.
Hatzenbuehler ML. Structural stigma: research evidence and implications for psychological science. Am Psychol. 2016;71:742.
Corrigan PW, Watson AC, Heyrman ML, Warpinski A, Gracia G, Slopen N, Hall LL. Structural stigma in state legislation. Psychiatr Serv. 2005;56:557–63.
National Conference of State Legislatures. Number of Legislators and Length of Term Years. Available from: http://www.ncsl.org/research/about-state-legislatures/number- oflegislators-and-length-of-terms.aspx. Accessed 10 Apr 2018.
Kirmayer LJ, Kronick R, Rousseau C. Advocacy as Key to Structural Competency in Psychiatry. JAMA Psychiat. 2018;75(2):119–20.
National Academies of Sciences E, Medicine: ending discrimination against people with mental and substance use disorders: the evidence for stigma change. Washington, DC: National Academies Press; 2016.
National Research Council. Preventing mental, emotional, and behavioral disorders among young people: Progress and possibilities. Washington, DC; National Academies Press; 2009.
Williamson A, Makkar SR, McGrath C, Redman S. How can the use of evidence in mental health policy be increased? A systematic review. Psychiatr Serv. 2015;66:783–97.
Purtle J, Peters R, Brownson RC. A review of policy dissemination and implementation research funded by the National Institutes of Health, 2007–2014. Implement Sci. 2016;11:1.
Hogan M. Using evidence to influence policy and improve practice. Psychiatr Serv. 2015;66:769.
Purtle, J., Dodson, E., Brownson, R. (2018). Policy dissemination research. In Dissemination and implementation research in health: translating science to practice. Ross C. Brownson, Graham A. Colditz, Enola K. Proctor (Eds.). 2nd Edition. New York: Oxford University Press.
Oliver K, Innvar S, Lorenc T, Woodman J, Thomas J. A systematic review of barriers to and facilitators of the use of evidence by policymakers. BMC Health Serv Res. 2014;14:2.
Goldner EM, Jeffries V, Bilsker D, Jenkins E, Menear M, Petermann L. Knowledge translation in mental health: a scoping review. Healthcare Policy. 2011;7:83.
Kennedy-Hendricks A, Barry CL, Gollust SE, Ensminger ME, Chisolm MS, McGinty EE. Social stigma toward persons with prescription opioid use disorder: associations with public support for punitive and public health–oriented policies. Psychiatr Serv. 2017;68:462–9.
McGinty E, Pescosolido B, Kennedy-Hendricks A, Barry CL. Communication strategies to counter stigma and improve mental illness and substance use disorder policy. Psychiatr Serv. 2017;69(2):136–46.
Barry CL, McGinty EE. Stigma and public support for parity and government spending on mental health: a 2013 national opinion survey. Psychiatr Serv. 2014;65:1265–8.
Barry CL, McGinty EE, Pescosolido BA, Goldman HH. Stigma, discrimination, treatment effectiveness, and policy: public views about drug addiction and mental illness. Psychiatr Serv. 2014;65:1269–72.
Stuber JP, Rocha A, Christian A, Link BG. Conceptions of mental illness: attitudes of mental health professionals and the general public. Psychiatr Serv. 2014;65:490–7.
McGinty EE, Goldman HH, Pescosolido BA, Barry CL. Communicating about mental illness and violence: balancing stigma and increased support for services. 2018;43(2):185–228. https://doi.org/10.1215/03616878-4303507.
Smith DM, Damschroder LJ, Kim SY, Ubel PA. What’s it worth? Public willingness to pay to avoid mental illnesses compared with general medical illnesses. Psychiatr Serv. 2012.
Maust DT, Moniz MH, Zivin K, Kales HC, Davis MM. Attitudes about required coverage of mental health care in a US national sample. Psychiatr Serv. 2015;66(10):1101–4.
Leeman J, Birken SA, Powell BJ, Rohweder C, Shea CM. Beyond “implementation strategies”: classifying the full range of strategies used in implementation science and practice. Implement Sci. 2017;12:125.
Slater MD. Theory and method in health audience segmentation. J Health Commun. 1996;1:267–84.
Kreuter MW, Bernhardt JM. Reframing the dissemination challenge: a marketing and distribution perspective. Am J Public Health. 2009;99:2123–7.
Kreuter MW, Farrell DW, Olevitch LR, Brennan LK. Tailoring health messages: Customizing communication with computer technology. Abingdon; Routledge; 2013.
Noar SM, Benac CN, Harris MS. Does tailoring matter? Meta-analytic review of tailored print health behavior change interventions. Psychol Bull. 2007;133:673.
Smith, RA. Audience segmentation techniques, Oxford research encyclopedia of communication. 2017.
Maibach E, Roser-Renouf C, Leiserowitz A: Global warming’s six Americas 2009: an audience segmentation analysis. 2009.
Arbuckle J, Tyndall J, Morton L, Hobbs J. Climate change typologies and audience segmentation among Corn Belt farmers. J Soil Water Conserv. 2017;72:205–14.
Hine DW, Reser JP, Morrison M, Phillips WJ, Nunn P, Cooksey R. Audience segmentation and climate change communication: conceptual and methodological considerations. Wiley Interdiscip Rev Clim Chang. 2014;5:441–59.
Nisbet MC, Maibach E, Leiserowitz A. Framing peak petroleum as a public health problem: audience research and participatory engagement in the United States. Am J Public Health. 2011;101:1620–6.
Poortinga W, Darnton A. Segmenting for sustainability: the development of a sustainability segmentation model from a Welsh sample. J Environ Psychol. 2016;45:221–32.
Bye L, Ghirardelli A, Fontes A. Promoting health equity and population health: how Americans’ views differ. Health Aff. 2016;35:1982–90.
Orton L, Lloyd-Williams F, Taylor-Robinson D, O'Flaherty M, Capewell S. The use of research evidence in public health decision making processes: systematic review. PLoS One. 2011;6:e21704.
Innvaer S, Vist G, Trommald M, Oxman A. Health policy-makers’ perceptions of their use of evidence: a systematic review. J Health Serv Res Pol. 2002;7:239–44.
Gollust SE, Seymour JW, Pany MJ, Goss A, Meisel ZF, Grande D. Mutual distrust: perspectives from researchers and policy makers on the research to policy gap in 2013 and recommendations for the future. J Health Care Organ Provision Financing. 2017;54:0046958017705465.
Brownson RC, Royer C, Ewing R, McBride TD. Researchers and policymakers: travelers in parallel universes. Am J Prev Med. 2006;30:164–72.
Cairney P, Oliver K. Evidence-based policymaking is not like evidence-based medicine, so how far should you go to bridge the divide between evidence and policy? Health Res Pol Systems. 2017;15:35.
Oliver K, Lorenc T, Innvær S. New directions in evidence-based policy research: a critical analysis of the literature. Health Res Pol Systems. 2014;12:34.
Purtle J, Lê-Scherban F, Shattuck P, Proctor EK, Brownson RC. An audience research study to disseminate evidence about comprehensive state mental health parity legislation to US State policymakers: protocol. Implement Sci. 2017;12:81.
Fisher SH III, Herrick R. Old versus new: the comparative efficiency of mail and internet surveys of state legislators. State Polit Pol Q. 2013;13:147–63.
Pagel C, Bates DW, Goldmann D, Koller CF. A way forward for bipartisan health reform? Democrat and republican state legislator priorities for the goals of health policy. Am J Public Health. 2017;107(10):1601–3.
Niederdeppe J, Roh S, Dreisbach C. How narrative focus and a statistical map shape health policy support among state legislators. Health Commun. 2016;31:242–55.
Zhu JM, Chhabra M, Grande, D. Concise Research Report: The Future of Medicaid: State Legislator Views on Policy Waivers. J Gen Intern Med. 1-3. 2018;33(7):999–1001. https://doi.org/10.1007/s11606-018-4432-8.
Holt D, Elliot D. "Methods of weighting for unit non-response." The Statistician. 1991;333–42.
Collins D. Pretesting survey instruments: an overview of cognitive methods. Qual Life Res. 2003;12:229–38.
Centers for Disease Control and Prevention, Substance Abuse and Mental Health Services Administration, National Association of County Behavioral Health & Developmental Disability Directors, National Institute of Mental Health, The Carter Center Mental Health Program. Attitudes toward mental illness: results from the Behavioral Risk Factor Surveillance System. Atlanta (GA): Centers for Disease Control and Prevention; 2012.
Borinstein AB. Public attitudes toward persons with mental illness. Health Aff. 1992;11:186–96.
Link BG, Phelan JC, Bresnahan M, Stueve A, Pescosolido BA. Public conceptions of mental illness: labels, causes, dangerousness, and social distance. Am J Public Health. 1999;89:1328–33.
Bogenschneider K, Corbett TJ. Evidence-based policymaking: insights from policy-minded researchers and research-minded policymakers. Abingdon: Routledge; 2011.
Purtle J, Dodson EA, Brownson RC. Uses of research evidence by State legislators who prioritize behavioral health issues. Psychiatr Serv. 2016;67:1355–61.
Brownson RC, Dodson EA, Kerner JF, Moreland-Russell S. Framing research for state policymakers who place a priority on cancer. Cancer Causes Control. 2016;27:1035–41.
Maibach EW, Leiserowitz A, Roser-Renouf C, Mertz C. Identifying like-minded audiences for global warming public engagement campaigns: an audience segmentation analysis and tool development. PLoS One. 2011;6:e17571.
Penn State Methodology Center. PROC LCA & PROC LTA (version 1.3.2). Methodology Center, Penn State: University Park, PA; 2015.
Collins LM, Lanza S. Latent class and latent transition analysis: with applications in the social, behavioral, and health sciences. Hoboken, NJ: John Wiley & Sons; 2013.
Lanza S, Dziak J, Huang L, Wangner A, Collins L. PROC LCA & PROC LTA users’ guide (version 1.3.2). The Methodology Center, Penn State: University Park, PA; 2015.
American National Ellection Studies. Guide to public opinion and electoral behavior. Liberal-Conservative Self-Identification 1972-2012. Available from: http://anesold.isr.umich.edu/nesguide/toptable/tab3_1.htm. Accessed 10 Apr 2018.
Brownson RC, Dodson EA, Stamatakis KA, Casey CM, Elliott MB, Luke DA, Wintrode CG, Kreuter MW. Communicating evidence-based information on cancer prevention to state-level policy makers. J Natl Cancer Inst. 2011;103(4):306-16.
Gronholm PC, Henderson C, Deb T, Thornicroft G. Interventions to reduce discrimination and stigma: the state of the art. Soc Psychiatry Psychiatr Epidemiol. 2017;52:249–58.
Haidt J: The righteous mind: why good people are divided by politics and religion. Vintage; 2012.
Jost JT, Federico CM, Napier JL. Political ideology: its structure, functions, and elective affinities. Annu Rev Psychol. 2009;60:307–37.
Chisholm D, Sweeny K, Sheehan P, Rasmussen B, Smit F, Cuijpers P, Saxena S. Scaling-up treatment of depression and anxiety: a global return on investment analysis. Lancet Psychiatry. 2016;3:415–24.
Florence CS, Zhou C, Luo F, Xu L. The economic burden of prescription opioid overdose, abuse, and dependence in the United States, 2013. Med Care. 2016;54:901–6.
Murphy SM, Polsky D. Economic evaluations of opioid use disorder interventions. PharmacoEconomics. 2016;34:863–87.
Schneider AL, Ingram HM. Deserving and entitled: social constructions and public policy. Albany: SUNY Press; 2005.
Oexle N, Corrigan PW. Understanding Mental Illness Stigma Toward Persons With Multiple Stigmatized Conditions: Implications of Intersectionality Theory. Psychiatr Serv. 2018;69(5):587–9.
Lavis JN, Lomas J, Hamid M, Sewankambo NK. Assessing country-level efforts to link research to action. Bull World Health Organ. 2006;84:620–8.
Washington State Institute for Public Policy. Benefit-Cost Results. http://www.wsipp.wa.gov/BenefitCost?topicId=8. Accessed 14 Sept 2018.
Purtle J, Henson RM, Carrol-Scott A, Kolker J, Diez Roux AVUS. Mayors’ evidence dissemination preferences: towards evidence-Based city policies. Implement Sci. 2018;13(Suppl 4):S104.
Crowley M, Scott JTB, Fishbein D. Translating prevention research for evidence-based policymaking: results from the research-to-policy collaboration pilot. Prev Sci. 2018;19:260–70.
Morshed AB, Dodson EA, Tabak RG, Brownson RC. Peer Reviewed: Comparison of Research Framing Preferences and Information Use of State Legislators and Advocates Involved in Cancer Control, United States, 2012–2013. Prev Chronic Dis, 14. 2017;14:E10. https://doi.org/10.5888/pcd14.160292.
Investigators C. Supporting Policy In health with Research: an Intervention Trial (SPIRIT)—protocol for a stepped wedge trial. BMJ Open. 2014;4:e005293.
Contandriopoulos D, Benoît F, Bryant-Lukosius D, Carrier A, Carter N, Deber R, Duhoux A, Greenhalgh T, Larouche C, Leclerc B-S. Structural analysis of health-relevant policy-making information exchange networks in Canada. Implement Sci. 2017;12:116.
Frank LB, Murphy ST, Chatterjee JS, Moran MB, Baezconde-Garbanati L. Telling stories, saving lives: creating narrative health messages. Health Commun. 2015;30:154–63.
Niederdeppe J, Heley K, Barry CL. Inoculation and narrative strategies in competitive framing of three health policy issues. J Commun. 2015;65:838–62.
Kennedy-Hendricks A, McGinty EE, Barry CL. Effects of competing narratives on public perceptions of opioid pain reliever addiction during pregnancy. J Health Polit Policy Law. 2016;41:873–916.
McGinty EE, Goldman HH, Pescosolido B, Barry CL. Portraying mental illness and drug addiction as treatable health conditions: effects of a randomized experiment on stigma and discrimination. Soc Sci Med. 2015;126:73–85.
McGinty EE, Webster DW, Barry CL. Effects of news media messages about mass shootings on attitudes toward persons with serious mental illness and public support for gun control policies. Am J Psychiatr. 2013;170:494–501.
Tudisca V, Valente A, Castellani T, Stahl T, Sandu P, Dulf D, Spitters H, Van de Goor I, Radl-Karimi C, Syed MA. Development of measurable indicators to enhance public health evidence-informed policy-making. Health Res Pol Sys. 2018;16:47.
Boyko JA, Lavis JN, Dobbins M, Souza NM. Reliability of a tool for measuring theory of planned behaviour constructs for use in evaluating research use in policymaking. Health Res Pol Sys. 2011;9:1.
Welch PJ, Dake JA, Price JH, Thompson AJ, Ubokudom SE. State legislators’ support for evidence-based obesity reduction policies. Prev Med. 2012;55:427–9.
Lyons RA, Kendrick D, Towner EM, Coupland C, Hayes M, Christie N, Sleney J, Jones S, Kimberlee R, Rodgers SE. The advocacy for pedestrian safety study: cluster randomised trial evaluating a political advocacy approach to reduce pedestrian injuries in deprived communities. PLoS One. 2013;8:e60158.
Butler DM, Nickerson DW. Can learning constituency opinion affect how legislators vote? Results from a field experiment. Q J Polit Sci. 2011;6:55–83.
Jason LA, Rose T. Influencing the passage of child passenger restraint legislation. Am J Community Psychol. 1984;12:485–94.
Gollust SE, Kite HA, Benning SJ, Callanan RA, Weisman SR, Nanney MS. Use of research evidence in state policymaking for childhood obesity prevention in Minnesota. Am J Public Health. 2014;104:1894–900.
Grossman G, Michelitch K. Information dissemination, competitive pressure, and politician performance between elections: a field experiment in Uganda. Am Polit Sci Rev. 2018;112:280–301.
Yanovitzky I, Weber M, Gesualdo N, Kristensen T: A Longitudinal Investigation of Knowledge Brokering As a Mechanism for Integrating Research Evidence into Health Policymaking.
Nyhan B, Reifler J. The effect of fact-checking on elites: a field experiment on US state legislators. Am J Polit Sci. 2015;59:628–40.
Grande D, Meisel ZF, Merchant RM, Seymour J, Gollust SE. Twitter accounts followed by congressional health staff. Am J Manag Care. 2017;23:e238–44.
CIPHER Investigators. Supporting Policy In health with Research: an Intervention Trial (SPIRIT)—protocol for a stepped wedge trial. BMJ Open. 2014;4(7):e005293. https://doi.org/10.1136/bmjopen-2014-005293.
Makkar SR, Williamson A, Turner T, Redman S, Louviere J. Using conjoint analysis to develop a system to score research engagement actions by health decision makers. Health research policy and systems. 2015;13(1):22.
Hoagwood KE, Olin SS, Horwitz S, McKay M, Cleek A, Gleacher A, Lewandowski E, Nadeem E, Acri M, Chor KHB. Scaling up evidence-based practices for children and families in New York state: toward evidence-based policies on implementation for state mental health systems. J Clin Child Adolesc Psychol. 2014;43:145–57.
Pescosolido BA, Jensen PS, Martin JK, Perry BL, Olafsdottir S, Fettes D. Public knowledge and assessment of child mental health problems: findings from the National Stigma Study-Children. J Am Acad Child Adolesc Psychiatry. 2008;47:339–49.
Pescosolido BA, Perry BL, Martin JK, McLeod JD, Jensen PS. Stigmatizing attitudes and beliefs about treatment and psychiatric medications for children with mental illness. Psychiatr Serv. 2007;58:613–8.
Perry BL, Pescosolido BA, Martin JK, McLeod JD, Jensen PS. Comparison of public attributions, attitudes, and stigma in regard to depression among children and adults. Psychiatr Serv. 2007;58:632–5.
Lanza ST, Rhoades BL. Latent class analysis: an alternative perspective on subgroup analysis in prevention and treatment. Prev Sci. 2013;14:157–68.
We thank the National Conference of State Legislatures for providing feedback on the survey instrument, SSRS for data collection, the legislators who participated in cognitive interviews to test the survey instrument, and the legislators who took the time to complete the survey.
Support for this project comes from the National Institute of Mental Health at the National Institutes of Health (R21MH111806; R25MH080916). The contents of this article are solely the responsibility of the authors and do not necessarily represent the official views of NIH.
Availability of data and materials
The datasets that result from the current study will be available from the corresponding author on reasonable request.
Ethics approval and consent to participate
All aspects of the study were approved by the Institutional Review Board of Drexel University (1608004754.)
Consent for publication
Not applicable. The manuscript does not contain any individual person’s data in any form.
The authors declare they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.