This article has Open Peer Review reports available.
Evaluation of a large healthy lifestyle program: informing program implementation and scale-up in the prevention of obesity
© The Author(s). 2016
Received: 20 April 2016
Accepted: 14 November 2016
Published: 24 November 2016
The Healthy Lifestyle Program for women (HeLP-her) is a low-intensity, self-management program which has demonstrated efficacy in preventing excess weight gain in women. However, little is known about the implementation, reach, and sustainability of low-intensity prevention programs in rural settings, where risk for obesity in women is higher than urban settings. We aimed to evaluate a low-intensity healthy lifestyle program delivered to women in a rural setting to inform development of effective community prevention programs.
A mixed method hybrid implementation and evaluation study, guided by the RE-AIM framework (addressing the Reach, Effectiveness, Adoption, Implementation, and Maintenance), was undertaken. Data collection tools included anthropometric measures, program checklists, questionnaires, and semi-structured interviews with participants and local stakeholders. The RE-AIM self-audit tool was applied to assess evaluation rigor.
Six hundred and forty-nine women from 41 relatively socio-economic disadvantaged communities in Australia participated: mean age 39.6 years (±SD 6.7) and body mass index of 28.8 kg/m2 (±SD 6.9). A between-group weight difference of −0.92 kg (95% CI −1.67 to −0.16) showed program effectiveness. Reach was broad across 41 towns with 62% of participants reporting influencing some of the health behaviors of their families. Strong implementation fidelity was achieved with good retention rates at 1 year (76%) and high participant satisfaction (82% of participants willing to recommend this program). Over 300 multi-level community partnerships were established supporting high adoption. Stakeholders reported potential capacity to implement and sustain the prevention program in resource poor rural settings, due to the low-intensity design and minimal resources required.
Our comprehensive RE-AIM evaluation demonstrates that an evidence-based obesity prevention program can be successfully implemented in real-world settings. The program achieved broad reach, effectiveness, and satisfaction at the community and stakeholder level, revealing potential for program sustainability. The evaluation addressed implementation knowledge gaps to support future obesity prevention program scale-up.
Treatment of established obesity via lifestyle interventions is challenging with poor engagement, high costs, and minimal sustainability [1, 2]. Weight loss is difficult to achieve at the individual level due to adaptive physiological responses post-weight loss, which almost universally drives weight regain [2, 3]. At the individual level, weight gain prevention is feasible, requiring only minor modifications to energy intake and expenditure . At the system level, primary prevention programs have the potential to reduce health care costs [1, 5]. In this context and with alarming escalation in obesity rates internationally, the World Health Organization (WHO) has identified prevention of excess weight gain as an international health priority .
Government prevention agencies have advocated the need for weight gain prevention programs in high-risk populations. Women have high rates of unhealthy weight gain [7, 8], and in many world regions, women have greater obesity prevalence than men  with higher subsequent obesity-related complications [10, 11]. Longitudinal data reveals that 20% of reproductive-aged women within a healthy weight range will become overweight within 5 years . For most individuals, weight gain is gradual over several decades and estimated at 600–800 g per year . Notably, even slight increases in weight of ~0.6 kg per year have been shown to increase the risk of breast cancer , hypertension , type 2 diabetes , and coronary heart disease  in women. Furthermore, as reproductive-aged women often have a key role in determining household food choices and sedentary behaviors, the need to invest in strategies to prevent excess weight gain in women is clear. In this context, we developed a low-intensity healthy lifestyle program for women, The Healthy Lifestyle Program for women (HeLP-her), shown to have efficacy for preventing weight gain in women in prior RCT’s in different settings and ethnically diverse populations [4, 17–19].
In developed countries, rural-dwelling women are more vulnerable than urban-dwelling women with lower socio-economic status and elevated rates of weight gain and obesity [20, 21]. Additional challenges include reduced access to primary health care services, resources, and trained health professionals . However, few healthy lifestyle programs have been implemented in rural settings [5, 23], and a systemic review has highlighted that the value of weight gain prevention programs in rural communities has not been established . As such, low-cost, low-intensity weight gain prevention programs are urgently needed in rural settings [20, 21].
Evaluation provides vital insights into how a program achieves efficacy and effectiveness. It also generates essential knowledge to drive implementation and scale-up [24–27] and deliver impact from research investment at the population level [28–32]. Despite this, few evaluations have been applied to weight gain prevention programs, leaving a major knowledge gap [24, 33, 34]. Common barriers to evaluation are lack of funding, time constraints, limited workforce knowledge, skills, and familiarity with evaluation methodologies and the dearth of valid evaluation tools .
The RE-AIM evaluation framework (Reach, Effectiveness, Adoption, Implementation, and Maintenance) explores program implementation and generalizability, focusing on the transferability of research findings into clinical practice and policy. The RE-AIM framework was developed specifically to evaluate health promotion interventions and encourages data collection at both the individual and organizational level [36, 37]. The RE-AIM framework has been applied numerous times to health promotion programs and to childhood obesity prevention programs [38, 39], highlighting its value and methodological rigor. However, on assessment of health promotion research programs and grants employing the RE-AIM framework, less than 10% had applied the complete RE-AIM reporting criteria [40, 41]. Most measured one or two RE-AIM dimensions only. The authors of the RE-AIM framework have recently developed criteria to assess the use of the RE-AIM framework .
The Healthy Lifestyle Program (HeLP-her) is an evidence-based weight gain prevention program for reproductive-aged women, which here was adapted for rural settings in the HeLP-her Rural trial, with an embedded hybrid implementation and evaluation design . This manuscript aims to (a) provide a summation of evaluation and implementation results from the large-scale HeLP-her Rural randomized controlled trial, utilizing the RE-AIM framework and (b) assess the application of the RE-AIM framework to the HeLP-her Rural program (utilizing the RE-AIM assessment criteria) , ultimately aiming to inform implementation and scale-up of obesity prevention programs broadly.
Program design, setting, theory, and implementation
The efficacy of the HeLP-her program has been established previously in two large RCT’s in a community setting for urban-dwelling women and in an antenatal clinic setting for pregnant women [4, 17]. Here, we have used an integrated community cluster RCT design to adapt and implement the HeLP-her program in relatively disadvantaged rural communities in the state of Victoria, Australia (HeLP-her Rural). As previously described, rural town selection was based on population size (2000–10,000 people) and distance from Melbourne central business district (CBD; towns located 100–400 km from CBD). Study randomization occurred at the town (cluster) level and analysis at the individual level. Overall, 41 rural townships met these criteria and were randomized to intervention or control towns. Randomization was conducted by the study biostatistician using a computer-generated randomization list. The primary outcome of HeLP-her Rural was the difference in weight gain between control and intervention groups at 12 months. Study methodology is published elsewhere .
The HeLP-her Rural program
HeLP-her Rural implementation and delivery
Regional government departments and community and school leaders were contacted by email, and a follow-up phone call was made. They were invited to support the program implementation by providing introductions to key community groups and assistance with recruitment and providing facilities for program delivery.
Program setting and facilitation
The program was facilitated by three tertiary qualified health professionals with expertise in nutrition, physical activity, and evidence-based practice, and all had worked within the Australian health sector previously. Program facilitators underwent a 1-day training day, led by the program leader, which covered the HeLP-her program theory and practical component, as well as provided motivational interviewing techniques.
Program theory and delivery
The program was designed to be low intensity and focused on participants making small long-term sustainable behavior changes. In this program, 41 rural communities were randomized to intervention or control groups. The control participants attended a single general group health information session. The intervention participants received lifestyle advice through mixed delivery modes including (i) limited personal contact: one group session and (ii) remotely, consisting of one phone coaching session, monthly text message reminders, and a program manual. The delivery methods were designed to reinforce program messages, appeal to various learning styles, and minimize program costs.
One 60-min group session was held with 8–15 women at community locations such as schools or halls. Facilitators delivered general health information plus simple health messages. Facilitators using an interactive model and supported by the program manual worked through examples of behavioral self-management skills including setting health priorities, problem solving, and self-monitoring, focusing on small changes to behavior.
The manual included simple information to improve knowledge and included activities to develop self-management skills such as problem solving, goal setting, and action planning. The participants completed the activities during the interactive group session and were then requested to work through manual activities in their own time.
Each participant was provided a single 20-min phone coaching session at 16 weeks post intervention commencement. The phone coaching session was delivered by trained coaches to assist completion of manual activities and reinforce program health messages.
SMS text messages and support
One text message was sent every 4 weeks in line with program messages, to remind the participants of the key program messages and goals.
Program evaluation design and theory
This manuscript reports on the results of the embedded program evaluation using an effectiveness-implementation hybrid study design and involved the simultaneous testing of clinical outcomes and implementation strategies. This research design is thought to facilitate “more rapid translational gains, more effective implementation strategies and promote the collection of useful information for decision makers for scale-up” . This study design is most appropriate for use when effectiveness of the program of interest has been established previously; such is the case for the HeLP-her program. The HeLP-her Rural evaluation aimed to explore program implementation strategies in general within complex systems, assessing implementation rigor. Within this evaluation, process evaluation measures included program fidelity, recruitment strategies, dose of the program delivered and received, program acceptability, and contextual factors influencing program implementation. A summative evaluation investigated overall program effectiveness, quality, community outcomes, and potential for future scale-up. Results on program effectiveness (weight changes), implementation (dose delivered and received, fidelity, recruitment, and program context) and the potential for program sustainability at the organization level have been previously published [46–50]. However, these results are also summarized here to address all elements of the RE-AIM framework.
Program recruitment and implementation
Data collection methods
Administrative data and contextual data
To explore program reach and context, data from the Australian Bureau of Statistics (ABS) measuring Socio-Economic Indexes for Areas (SEIFA) of relative disadvantage was utilized . Potential scores ranged from 1 to 10 with a lower score indicating a greater level of social disadvantage relating to household total income, education attained, and unemployment rates.
Program checklists and log books
Program specific process evaluation checklists were developed in conjunction with the research team field notes to assess implementation fidelity, recruitment, dose delivered, and program context. Program log books were kept to monitor program communication with organizational program partners.
Observations by the research team
The intervention was delivered by a research team (dieticians and exercise physiologists), working in pairs, with one researcher delivering the intervention, the other observing implementation, collecting data, completing checklists and offering feedback to ensure delivery consistency. Program fidelity was addressed by all researchers undergoing a 1-day training workshop (August 2012) delivered by an experienced trainer (CB) and receiving ongoing support and utilizing program developed presentations and resources, promoting consistency of program delivery (Table 1).
Weight, height, and waist and hip circumference measurements were collected by the trained team at baseline, 12 months (end of active intervention), and 24 months (end of observation).
Questionnaires completed by participants
All participants completed program devised questionnaires at baseline (prior to intervention commencement) and 12 and 24 months including items on demographic characteristics, socio-cultural and physical environment, health status, and program satisfaction. Participant program satisfaction (overall program, information provided, program delivery, and support provided) was assessed on a program-devised Likert scale (1–5) with higher scores representing greater satisfaction.
Semi-structured (qualitative) participant and stakeholder interviews
In-depth semi-structured interviews were conducted with a sub-group of participant’s 6 months post intervention commencement. A criteria-based, convenience sampling approach as previously described [47, 48] involved women from ten communities (six intervention and four control). One trained researcher conducted all interviews (SK), guided by an interview schedule (Additional file 2). Forty-five participant interviews were conducted until data saturation was met, determined when no new ideas emerged from the interviews, as per standard methods .
Local stakeholders were identified as those who had a direct interest in the development, delivery, and outcomes of prevention programs and included local government employees, clinical health care providers (general practitioners, allied health, community nurses), and non-clinical health providers as previously described . These stakeholders had not directly implemented the HeLP-her Rural program, rather enabled access to local networks for the research team during program implementation. Stakeholders were asked to provide insight into the enablers, barriers, and strategies that would promote program implementation for weight gain prevention. Prior to interviews, each stakeholder was provided with an overview of the HeLP-her study background, the design, staffing, intensity, and implementation plan. Twenty-four semi-structured telephone interviews were conducted led by one researcher (SK) (Additional file 3).
Evaluation self-audit (meta-evaluation) of the HeLP-her Rural program
A meta-evaluation refers to the auditing and assessment of a program evaluation, determining if the evaluation has produced credible and justifiable conclusions . We applied a criteria-based tool to assess the application of the RE-AIM framework to the HeLP-her Rural evaluation, developed by Kessler et al. . This tool outlines a minimum set of “core” items (n = 31) producing total and item-specific scores for each of the five RE-AIM domains. This corresponds to a comprehensive application of the framework, termed “fully developed use of RE-AIM”. Items that are not applicable to the program were excluded from the total calculated score (Table 2) .
Meta-evaluation of HeLP-her Rural using criteria developed by RE-AIM
A. Participant exclusion criteria (% excluded)
Based on predefined exclusion criteria, less than 12% of the participants (n = 95) were excluded post screening (Fig. 1).
B. Percentage who participate
We recruited 649 women into the HeLP-her program or ~10% n of the potential target population.
C. Participants characteristics versus nonparticipants
The women involved were representative of the broader Australian regional population (income and education).
D. Qualitative methods
We qualitatively explored program reach (Fig. 2).
Scoring: “Fully Developed Use”
1. (B) and (C) and at least one other item (A or D)
=Fully Developed Use (A + B + C + D): total of (4/4)
A. Primary outcome measures
At 1-year, the mean weight change in controls was +0.44 kg and in intervention groups was −0.48 kg, a between group difference of −0.92 kg (95% CI −1.67 to −0.16).
B. Measure of broader outcomes
A broad range of outcomes are described elsewhere (food intake, physical activity, self-efficacy, quality of life)
C. Robustness across sub-groups
The intervention showed equally efficacy across various age, BMI, income, and education sub-groups.
D. Attrition (%)
The study retention was 76% at 1 year (Fig. 1).
E. Qualitative methods
Program effectiveness was explored qualitatively.
Scoring: Fully Developed Use
1. Has (A), (B), (C), and (D)
=Fully Developed Use (A + B + C + D + E): total of 5/5 “Yes”
Adoption (setting level)
A. Setting exclusions (% or reasons)
Yes, one control town was excluded due to difficultly with participant recruitment. This was because recruitment was conducted during peak farming times “harvesting” (Fig. 1).
B. Percentage of settings approached that participated
We contacted 311 local stakeholders and 95% (n = 311) agreed to partner with the HeLP-her program, assisting implementation (Table 3).
C. Characteristics of settings participating versus nonparticipation
Not explored. However, township selection was based on randomization techniques.
D. Use of qualitative methods
Semi-structured stakeholder interviews were conducted.
Scoring: “Fully Developed Use”—adoption setting
1. Must have (B) and (C) and at least one other item (A or D)
=Partially Developed (A + B + D + E): total score of 4/5
Adoption-staff level—not applicable
Scoring: “Fully Developed Use”—adoption-researchers
A. Percentage of full delivery or full calls
Comprehensive process evaluation results revealed strong implementation fidelity and high dose delivered.
A. Program adaptions
Implementation was standardized across communities as per study protocol with minor adaptations reported previously.
B. Cost of intervention
Comprehensive economic evaluation is underway.
C. Consistency of researchers, time, and setting
Comprehensive process evaluation indicated implementation consistency.
D. Qualitative methods applied
Program implementation was explored at the community and organizational level with high program acceptability
Scoring: “Fully Developed Use”—implementation:
1. Have (A), (C), and (D) plus at least one more item (B or E)
Fully Developed Use = (A + B + C + D + E): total of 5/5
A. Primary outcome after final intervention
As above, anthropometric data was collected at baseline and 12 and 24 months with results pending.
B. Measure of broader outcomes, multiple criteria at follow-up
Data analysis collected at 0 and 12 months with food intake, physical activity, self-efficacy and self-management. These outcomes measures will be again explored at 24 months.
C. Robustness data—sub-group effects over the long term
24-month data analysis planned with results pending.
D. Attrition (%)
24-month data analysis planned with results pending.
Scoring: “Fully Developed Use”—maintenance—individual: has (A), (B), (C), and (D)
Fully Developed Use = (A) + (B) + (C) + (D): total of 4/4
A. Program continuation 6 months post study completion
The HeLP-her program has been endorsed by the Victoria local government preventative health taskforce
B. How program was adapted
C. Discussion of alignment to organization mission
Exploration undertaken with stakeholders, highlighting that prevention orientated program aligns with local organizational values.
D. Use of qualitative methods.
Stakeholder interviews conducted exploring potential for program continuation and “scale-up”.
Scoring: “fully developed use”—maintenance-setting
1. Has (A) and at least 1 more item (B, C, or D)
Fully Developed Use = (A) + (C) + (D) = 3/3
Entire RE-AIM model scoring
Fully Developed Use = (A + B + C + D): total of 4/4 “Yes”
Fully Developed Use = (A + B + C + D + E): total of 5/5 “Yes”
Partial Developed (A + B + D + E): total of 4/5 “Yes”
Fully Developed Use = (A + B + C + D + E): total of 5/5 “Yes”
Fully Developed Use = (A) + (B) + (C) + (D): total of 4/4 “Yes”
Fully Developed Use = (A) + (C) + (D): total of 3/3 “Yes”
Total score: 25/26 = 96% across all RE-AIM dimensions
The statistical analysis plan has been previously reported as has statistical methods applied [42, 49]. Data analysis was conducted using STATA and SPSS version 19.0 for Windows. The trial was designed to have a statistical power of 80% to detect a difference of 1.0 kg in weight between groups at 1-year with the use of a two-sided test at a significance level of 0.05. Results are presented as mean (SD) for continuous and relative frequencies for categorical data.
Qualitative transcripts were analyzed thematically, with grounded theory principles of analysis enabling the identification, coding, and categorization of primary data patterns. All transcripts were independently analyzed and coded by two investigators, assisted by the NVivo Software program (QSR International Pty Ltd. Version 10, 2012, Victoria, Melbourne).
Q1: To what extent did the program reach the target group?
The baseline age and BMI of participants was 39.6 ± 6.7 years and 28.8 ± 6.9 kg/m2, respectively. The participants had diverse education levels, household sizes, and income levels and were representative of the broader Victorian regional and rural population for women of a similar age (25–54 years) for income and education [50, 54, 55].
Q2. What were the key motivators and barriers to program engagement at the participant level (reach)?
As previously described in qualitative analysis, motivators for program participation were convenience of the program location and perceived program utility such as weight management and optimisation of lifestyle choices, as well as attending the program with peers. Barriers to engagement included lack of anonymity, self-consciousness, and segregated social networks in rural settings .
Q3: Are healthy lifestyles contagious amongst family members and the social networks of program participants?
It’s a roll on effect because if you’re eating healthy yourself and doing healthy meals it follows on to the kids – [intervention participant]
(My husband) he’s my resistance band. He’s like, no, you can have Coke, you can have pies. It’s very hard (for me)- [control participant]
I’ve got two kids that are very fussy eaters…At the end of the day I can’t be bothered with the arguments and whinging - [control participant]
My weight is something I’m very embarrassed about so I tend not to discuss it with friends. I don’t suppose my friends are struggling with it as I am - [control participant]
Social barriers to broader program reach included lack of socially acceptability to talk to friends about their weight and lifestyle choices as they do not want to be perceived as “self-absorbed” and “talking about myself all the time.” Other participants described that as many of their friends were already “very active” and “pretty healthy,” there was no need to discuss program learnings. Social isolation was another commonly described reason for not influencing social networks. Others noted beliefs that their friends were “unsupportive” and disinterested, “none of my friends are really into exercise and healthy lifestyles”.
Consistent with our qualitative findings, quantitative questionnaire data collected from intervention (n = 230) and control participants (n = 207) at 12 months showed that 62% of the participants reported influencing health behaviors of their families. However, there was no statistical significant difference between groups. The participants were most likely to influence their children (38%), friends (34%), and partners (30%).
Q4. Was the HeLP-her Rural program efficacious at preventing weight gain in intervention participants?
As previously reported, HeLP-her Rural prevented weight gain at 1 year with the mean weight change from baseline in the control group at +0.44 kg (95% CI −0.09 to 0.97) and the intervention group at −0.48 kg (95% CI −0.99 to 0.03), with a between-group difference of −0.92 kg (95% CI −1.67 to −0.16) or −0.88 kg (95% CI −1.62 to −0.13) adjusted for baseline values and clustering .
Q5. What were the enablers and barriers to behavior change sustainability within this program?
Behavior change continuation was facilitated by participant’s ability to apply the core program messages such as setting achievable behavior change goals, problem solving, and relapse prevention. Improved health knowledge, internal motivation, self-efficacy, and internal accountability all supported continued behavior change as previously described .
Q7. How many and which organizations supported HeLP-her Rural program implementation?
HeLP-her Rural partnership development
Types of organizations
Number of organizations contacted
Number of organizations that supported HeLP-her rural
Success rate of partnerships developed (%)
Local government employees
●Local government area managers
●Community development officers
Primary and Catholic schools
Kindergartens/child care centers
●Gym owners/personal trainers
Health centers/primary Care
●Health service managers
●Health promotion officers
●Clinical researchers (GP’s, nurses, allied health)
●Rural women’s organizations
Q8. Do stakeholders value weight gain prevention programs?
I would encourage my managers to support [prevention programs] because, the demand on the service for weight management is huge and it is particularly women - [Local dietician]
The model (program theory) sounded like something worth promoting. That low commitment program… It sounded like a good program to support – [Health Service Manager].
In addition, many stakeholders valued the use of remote methods to deliver lifestyle advice, “It’s new, innovative ideas and I think that using digital medium is a great way.”
I think we could probably deliver a program like that ourselves, with the correct information, I mean and tailoring it to what we’ve got. Actually I was sitting in on the program and thinking you know we could do this –[Community nurse]
Stakeholders provided recommendations to improve program delivery and optimize future scale-up of prevention programs. As previously reported, key recommendations included the development of multi-level partnerships, ongoing mentoring relationships via electronic communication, delivery of programs amongst outlying rural townships, and the provision of a suite of implementation resources to support cost and time-efficient implementation .
Q9. To what extent had the HeLP-her Rural program been implemented as per the study protocol?
Process evaluation revealed strong implementation fidelity and high dose delivered and received, confirmed through administration records, researchers observations, and completed program devised checklists, as previously reported. This highlighted the acceptability of low-intensity healthy lifestyle programs with mixed face-to-face and remote delivery modes in this population group. Qualitative participant interviews revealed that group education sessions were most valued, followed by text messages and phone coaching. Overall, delivery of lifestyle advice through multiple delivery modes (group sessions, phone coaching, text messages, and a program manual) was recommended by participants to optimize program acceptability and accommodate diverse learning styles . A full economic analysis is underway based on a pending 2-year data analysis and will be reported elsewhere.
Q10. Were participants satisfied with the HeLP-her Rural program experience?
I think [the program] is fantastic and just like the fact that you held it in the first place, I thought oh wow, there’s people that are actually out there to help and yes, it felt like a privilege – [intervention participant]
The 1-year satisfaction surveys showed high participant satisfaction with 82% of the participants agreeing they would recommend this program, with more intervention participants recommending the program (90 versus 73%, P < 0.05) and being satisfied with program support than the controls (84 versus 61%, P < 0.05).
Q11. Is there evidence of an organizational demand for program continuation post study completion?
The HeLP-her program has been identified as an evidence-based community program by the Victorian State Government as part of a state-wide system-based approach to tackling the rising rates of obesity and preventing obesity-related conditions . This demonstrates the value of a self-management lifestyle program to prevent weight gain and the need for comprehensive evaluation and implementation evidence.
Q12. Utilizing a criteria-based meta-evaluation audit tool, does the HeLP-her Rural program measure all “core” RE-AIM dimensions?
Employing Kessler’s et al. RE-AIM meta-evaluation tool, this evaluation assessed all “core” elements of the RE-AIM framework, producing a score of 96% for “fully developed use across all RE-AIM dimensions” (Table 2) . We were unable to assess the domain “adoption at the researcher and staff level” as this was not applicable.
The HeLP-her, low-intensity healthy lifestyle program has now been delivered in a range of settings and populations in a series of RCT’s where it effectively improves lifestyle and prevents weight gain [4, 17, 49]. The HeLP-her Rural program achieved broad program reach across 41 rural townships with most participants reporting influencing some of the health behaviors of their families. Program implementation was supported by 300 multi-level partnerships with partners valuing program theory and low-intensity program design. Stakeholders reported capacity to locally implement and sustain the program. The HeLP-her Rural program prevented a weight gain of nearly 1 kg on average amongst women living in rural Australia. The US Agency for Healthcare Research and Quality defines a weight difference of 0.5 kg between groups as clinically significant and meaningful. Notably, a modeling study estimated that a 1-kg weight loss, if applied across the USA population, could avoid 2 million cases of diabetes, 1.5 million cases of cardiovascular disease, and more than 73,000 cases of cancer . These findings add new information on effective weight gain prevention strategies, mirroring current clinical guidelines and addressing international health priorities to halt the obesity epidemic.
The RE-AIM framework was developed to facilitate translation of research findings into improved population health outcomes, focusing on increasing the reporting of implementation strategies and external program validity . A systematic review of the RE-AIM framework shows broad application to health promotion programs mainly around chronic disease. However, the framework has been inconsistently and sometimes incorrectly applied across all five dimensions [40, 41]. Reporting all RE-AIM dimensions and criteria is critical with interdependent relationships important for public health impact . Updated RE-AIM criteria  recommend qualitative research to improve understanding of results, yet this is rarely employed [40, 41]. Here, we address key evaluation and obesity lifestyle prevention gaps by applying qualitative and quantitative research methods and extending evaluation to all RE-AIM dimensions. Subsequently, increasing the generalizability of our results to obesity prevention programs more broadly.
Here, we targeted rural women based on high rates of weight gain, risk of obesity and related complications, adverse impact on maternal and child short- and long-term health, and relative disadvantage for rural women. Despite reported difficulties in engaging socio-economic disadvantaged groups into research , we reached women from disadvantaged communities who were representative of regional women of a similar age (25–54 years) based on income, household size, and education . We note that participants reported directly influencing their household’s lifestyle behaviors, such as improving fruit and vegetable intake and physical activity frequency and reducing discretionary food and takeaway consumption. Research exploring the relationship between lifestyle changes in women participating in healthy lifestyle programs and family reach is limited. Further research exploring measurable impact of participant’s reports of influence with members of the household is needed. However, our results support the potential importance of targeting reproductive-aged women [59–61], due to their influential role in influencing household lifestyle behaviors and food choices .
The HeLP-her Rural program and evaluation has addressed the dearth of literature surrounding the acceptability of low-intensity weight gain prevention programs in young women [59, 63]. The program has successfully engaged 1127 women across three large RCT’s, emphasizing that low-intensity programs appeal to women from highly diverse socio-economic, ethnic, and educational backgrounds [4, 17, 47]. Furthermore, we have shown that a combination of delivery modes (face-to-face, phone coaching, text messages, and program manuals) optimizes program acceptability and delivers effectiveness. This is in keeping with a recent systematic review reporting the effectiveness of using diverse e-health methods (emailing, texting, phone apps, and websites) to prevent and manage obesity . Supporting the high value of weight gain prevention programs in women, 92% of intervention participants reported that they would recommend this program. Conversely, intensive weight loss programs are generally poorly received with low engagement and high attrition rates . The value and effectiveness of low-intensity weight gain prevention programs demonstrated here can inform the design and execution of prevention programs and strategies more broadly.
Implementing, scaling up, and sustaining evidence-based lifestyle programs in diverse real-world settings are notoriously challenging, yet they are fundamental to improve public health outcomes . Here, we report on the strong potential for program adoption and scale-up, related to broad community reach, effectiveness, low-intensity program design, and high acceptability at the participant and organizational levels. Supporting the value of the prevention-based programs at the organizational level, we successful partnered with local organizations. Stakeholders valued the underlying program theory and design and alignment of the program objectives with their organizational health priorities because this was a unique professional development opportunity. These partnerships were essential to optimizing program implementation, especially in relation to participant engagement. As previously reported , stakeholders recommended strategies for optimizing program sustainability and scale-up including building local capacity, developing partnerships and minimizing implementation costs [67, 68]. Moving forward, the HeLP-her program is ready for delivery by local rural health professionals with an interest in weight gain prevention. To date, training of rural communities by the HeLP-her Rural research team has occurred to support professional capacity building of the rural workforce. Funding has been provided by the Australian government preventative health taskforce to refine resources, engage, and adapt considering ethnic diversity and for initial steps in community roll-out . However, to enable program sustainability long-term funding is required, and partnerships with funders in a range of setting are currently in development.
Study strengths and limitations
Study strengths include utilizing a robust mixed methods evaluation design with an underpinning theoretical framework and a meta-evaluation that revealed all “core” RE-AIM elements were assessed. However, we note that “adoption” at the staff level was unable to be measured within this context due to program design. Limitations include that our program checklists exploring program fidelity and contextual influences were completed by researchers involved in the trial, rather than independent evaluators. Contextual factors and characteristics could also have been further explored. Furthermore, whilst comprehensive 2-year weight data and program economic analysis are underway, these data are needed to inform program maintenance. Given a large amount of data needed to meet all “core” RE-AIM dimensions, a staged approach to data collection and analysis in RE-AIM evaluation is common.
Evidence-based obesity prevention programs such as the HeLP-her healthy lifestyle program have potential for population level scale-up, with broad reach beyond direct participants, demonstrated effectiveness across a range of populations and settings, low-intensity design, and high acceptability at the community and stakeholder levels. Here, we report strong program implementation rigor and comprehensive application of the RE-AIM framework. Moving forward, there is a clear need to redefine lifestyle program success if we are to deliver public health impact. Success must extend beyond effectiveness to incorporate program reach, adoption and potential for implementation and sustainability in diverse real-world settings. This is critical to inform policy and scale-up of lifestyle programs to address the current obesity epidemic.
The authors would like to thank all participants and stakeholders that gave up their time to participate in this study and provided valuable insights. We would also like to acknowledge Kim Hider for her assistance with designing the HeLP-her evaluation framework and Dr Lisa Moran for her input. Thank you to Dr. Nicole Ng, Rachel Miller, and Kelly Ashcroft for assistance with participant recruitment, data collection, and collation. Samantha Kozica is an NHMRC postgraduate scholar, Cheryce Harrison is a National Heart Foundation Postdoctoral Research Fellow (100168), and Helena Teede is an NHMRC Practitioner Fellow.
This trial was funded by a project grant from the National Health and Medical Research Council Australia (ID 1022951). The funders had no role in the study design, the data collection and analysis, the decision to publish, or the preparation of the manuscript.
Availability of data and materials
All HeLP-her data files are available from the Figshare database at the following link: http://dx.doi.org/10.6084/m9.figshare.1618679. Participant and stakeholder qualitative interview transcripts are available upon request.
CL and HT conceptualized the RCT. All authors provided intellectual input into the evaluation design and methodology. SK conducted the interviews, analyzed the interviews, and drafted the manuscript. All authors contributed to, reviewed, and approved the manuscript.
All authors declare that they have no competing interests.
Ethics approval and consent to participate
The study and embedded implementation-evaluation program were approved by the Monash Health Research Ethics Committee for research involving humans, and all participants provided written informed consent, project No. 12034B. Written consent was provided by all participants.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Lemmens VE, Oenema A, Klepp KI, Henriksen HB, Brug J. A systematic review of the evidence regarding efficacy of obesity prevention interventions among adults. Obes Rev. 2008;9(5):446–55.View ArticlePubMedGoogle Scholar
- Apovian CM, Aronne LJ, Bessesen DH, McDonnell ME, Murad MH, Pagotto U, Ryan DH, Still CD. Pharmacological management of obesity: an endocrine society clinical practice guideline. J Clin Endocrinol Metab. 2015;100(2):342–62.View ArticlePubMedGoogle Scholar
- Hinkle W, Cordell M, Leibel R, Rosenbaum M, Hirsch J. Effects of reduced weight maintenance and leptin repletion on functional connectivity of the hypothalamus in obese humans. PLoS One. 2013;8(3):e59114.View ArticlePubMedPubMed CentralGoogle Scholar
- Lombard C, Deeks A, Jolley D, Ball K, Teede H. A low intensity, community based lifestyle programme to prevent weight gain in women with young children: cluster randomised controlled trial. BMJ. 2010;341:c3215.View ArticlePubMedPubMed CentralGoogle Scholar
- Lombard CB, Deeks AA, Teede HJ. A systematic review of interventions aimed at the prevention of weight gain in adults. Public Health Nutr. 2009;12(11):2236–46.View ArticlePubMedGoogle Scholar
- World Health Organization. Obesity: preventing and managing the global epidemic. In: WHO Technical Report Series 894. Avaialble: http://www.who.int/nutrition/publications/obesity/WHO_TRS_894/en/. Accessed 3 Apr 2012.
- Australian Government National Preventative Health Taskforce. Australia: the healthiest country by 2020. In. Avaialble: http://www.preventativehealth.org.au/internet/preventativehealth/publishing.nsf/Content/nphs-roadmap-toc. Accessed 28 Jun 2012.
- Australian Longitudinal Study on Women’s Health. Australian women and their weight—a growing problem 2005 In. Available: http://www.alswh.org.au/. Accessed 4 Jan 2012.
- Alwan A. Global status report on noncommunicable diseases 2010. In: World Health Organization. Available: http://www.who.int/nmh/publications/ncd_report2010/en/. Accessed 6 Mar 2015.
- Guelinckx I, Devlieger R, Beckers K, Vansant G. Maternal obesity: pregnancy complications, gestational weight gain and nutrition. Obes Rev. 2008;9(2):140–50.View ArticlePubMedGoogle Scholar
- Hutchesson MJ, Hulst J, Collins CE. Weight management interventions targeting young women: a systematic review. J Acad Nutr Diet. 2013;113(6):795–802.View ArticlePubMedGoogle Scholar
- Andajani-Sutjahjo S, Ball K, Warren N, Inglis V, Crawford D. Perceived personal, social and environmental barriers to weight maintenance among young women: a community survey. Int J Behav Nutr Phys Act. 2004;1(1):15.View ArticlePubMedPubMed CentralGoogle Scholar
- Trentham-Dietz A, Newcomb PA, Egan KM, Titus-Ernstoff L, Baron JA, Storer BE, Stampfer M, Willett WC. Weight change and risk of postmenopausal breast cancer (United States). Cancer Causes Control. 2000;11(6):533–42.View ArticlePubMedGoogle Scholar
- Huang Z, Willett WC, Manson JE, Rosner B, Stampfer MJ, Speizer FE, Colditz GA. Body weight, weight change, and risk for hypertension in women. Ann Intern Med. 1998;128(2):81–8.View ArticlePubMedGoogle Scholar
- Colditz GA, Willett WC, Rotnitzky A, Manson JE. Weight gain as a risk factor for clinical diabetes mellitus in women. Ann Intern Med. 1995;122(7):481–6.View ArticlePubMedGoogle Scholar
- Willett WC, Manson JE, Stampfer MJ, Colditz GA, Rosner B, Speizer FE, Hennekens CH. Weight, weight change, and coronary heart disease in women. Risk within the ‘normal’ weight range. JAMA. 1995;273(6):461–5.View ArticlePubMedGoogle Scholar
- Harrison CL, Lombard CB, Strauss BJ, Teede HJ. Optimizing healthy gestational weight gain in women at high risk of gestational diabetes: a randomized controlled trial. Obesity. 2013;21(5):904–9.View ArticlePubMedGoogle Scholar
- Lombard CB, Deeks AA, Ball K, Jolley D, Teede HJ. Weight, physical activity and dietary behavior change in young mothers: short term results of the HeLP-her cluster randomized controlled trial. Nutr J. 2009;8:17.View ArticlePubMedPubMed CentralGoogle Scholar
- Harrison CL, Lombard CB, Teede HJ. Limiting postpartum weight retention through early antenatal intervention: the HeLP-her randomised controlled trial. Int J Behav Nutr Phys Act. 2014;11(1):1.View ArticleGoogle Scholar
- Janus ED, Laatikainen T, Dunbar JA, Kilkkinen A, Bunker SJ, Philpot B, Tideman PA, Tirimacco R, Heistaro S. Overweight, obesity and metabolic syndrome in rural southeastern Australia. Med J Aust. 2007;187(3):147–52.PubMedGoogle Scholar
- Cleland V, Hume C, Crawford D, Timperio A, Hesketh K, Baur L, Welch N, Salmon J, Ball K. Urban-rural comparison of weight status among women and children living in socioeconomically disadvantaged neighbourhoods. Med J Aust. 2010;192(3):137–48.PubMedGoogle Scholar
- Wakerman J, Humphreys JS, Wells R, Kuipers P, Entwistle P, Jones J. Primary health care delivery models in rural and remote Australia—a systematic review. BMC Health Serv Res. 2008;8(1):276.View ArticlePubMedPubMed CentralGoogle Scholar
- Lemmens VEPP, Oenema A, Klepp K, Henriksen HB, Brug J. A systematic review of the evidence regarding efficacy of obesity prevention interventions among adults. Obes Rev. 2008;9(5):446–55.View ArticlePubMedGoogle Scholar
- Partridge S, Juan SH, McGeechan K, Bauman A, Allman-Farinelli M. Poor quality of external validity reporting limits generalizability of overweight and/or obesity lifestyle prevention interventions in young adults: a systematic review. Obes Rev. 2015;16(1):13–31.View ArticlePubMedGoogle Scholar
- Oldenburg B, Absetz P. Lost in translation: overcoming the barriers to global implementation and exchange of behavioral medicine evidence. Transl Behavioral Med. 2011;1(2):252–5.View ArticleGoogle Scholar
- Fixsen DL, Naoom SF, Blase KA, Friedman RM, Wallace F. Implementation Research: A Synthesis of the Literature. Tampa: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231); 2005.Google Scholar
- Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE. Knowledge translation of research findings. Implement Sci. 2012;7(1):5060.View ArticleGoogle Scholar
- Baranowski T, Stables G. Process evaluations of the 5-a-day projects. Health Educ Behav. 2000;27(2):157–66.View ArticlePubMedGoogle Scholar
- Saunders RP, Evans MH, Joshi P. Developing a process-evaluation plan for assessing health promotion program implementation: a how-to guide. Health Promot Pract. 2005;6(2):134–47.View ArticlePubMedGoogle Scholar
- Grant A, Treweek S, Dreischulte T, Foy R, Guthrie B. Process evaluations for cluster-randomised trials of complex interventions: a proposed framework for design and reporting. Trials. 2013;14(1):15–25.View ArticlePubMedPubMed CentralGoogle Scholar
- Burchett H, Umoquit M, Dobrow M. How do we know when research from one setting can be useful in another? A review of external validity, applicability and transferability frameworks. J Health Serv Res Policy. 2011;16(4):238–44.View ArticlePubMedGoogle Scholar
- Patel B, Patel A, Usherwood J, Stephen T, Harris M, Peiris D. A multifaceted quality improvement intervention for CVD risk management in Australian primary healthcare: a protocol for a process evaluation. Implement Sci. 2014;9:187–98.View ArticlePubMedPubMed CentralGoogle Scholar
- Compernolle S, De Cocker K, Lakerveld J, Mackenbach J, Nijpels G, Oppert J-M, Rutter H, Texeira P, Cardon G, De Bourdeaudhuij I. A RE-AIM evaluation of evidence-based multi-level interventions to prevent obesity in adults: a systematic review. Int J Behav Nutr Phys Act (ISBNPA). 2014. (in press).Google Scholar
- Kremers S, Reubsaet A, Martens M, Gerards S, Jonkers R, Candel M, de Weerdt I, de Vries N. Systematic prevention of overweight and obesity in adults: a qualitative and quantitative literature analysis. Obes Rev. 2010;11(5):371–9.View ArticlePubMedGoogle Scholar
- Kozica SL, Lombard CB, Harrison CL, Hider K, Teede HJ. Developing comprehensive health promotion evaluations: a methodological review. MedCrave Online J Public Health. 2015;2(1):1–7.Google Scholar
- Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89(9):1322–7.View ArticlePubMedPubMed CentralGoogle Scholar
- McKenzie R, Naccarella L, Stewart A, Thompson C. Targeting what matters in health promotion evaluation. Eval J Australasia. 2007;7(1):19–26.Google Scholar
- Jenkinson KA, Naughton G, Benson AC. The GLAMA (Girls! Lead! Achieve! Mentor! Activate!) physical activity and peer leadership intervention pilot project: a process evaluation using the RE-AIM framework. BMC Public Health. 2012;12:55.View ArticlePubMedPubMed CentralGoogle Scholar
- De Meij JS, Chinapaw MJ, Kremers SP, Van der Wal MF, Jurg ME, Van Mechelen W. Promoting physical activity in children: the stepwise development of the primary school-based JUMP-in intervention applying the RE-AIM evaluation framework. Br J Sports Med. 2010;44(12):879–87.View ArticlePubMedGoogle Scholar
- Kessler RS, Purcell EP, Glasgow RE, Klesges LM, Benkeser RM, Peek C. What does It mean to “employ” the RE-AIM model? Eval Health Prof. 2012;36(1):44–66.View ArticlePubMedGoogle Scholar
- Gaglio B, Shoup JA, Glasgow RE. The RE-AIM framework: a systematic review of use over time. Am J Public Health. 2013;103(6):e38–46.View ArticlePubMedPubMed CentralGoogle Scholar
- Lombard C, Harrison C, Kozica S, Ng N, Teede H. Translation and implementation of an evidence based program to prevent weight gain in women. Obes Res Clin Pract. 2013;7:e76.View ArticleGoogle Scholar
- Deci EL, Ryan RM. Self determination. London: Wiley Online Library; 1985.Google Scholar
- Rubak S, Sandbæk A, Lauritzen T, Christensen B. Motivational interviewing: a systematic review and meta-analysis. Br J Gen Pract. 2005;55(513):305–12.PubMedPubMed CentralGoogle Scholar
- Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012;50(3):217–26.View ArticlePubMedPubMed CentralGoogle Scholar
- Kozica SL, Teede HJ, Harrison CL, Klein R, Lombard CB. Optimizing implementation of obesity prevention programs: a qualitative investigation within a large-scale randomized controlled trial. J Rural Health. 2015. (In press).Google Scholar
- Kozica SL, Lombard CB, Harrison CL, Ilic D, Ng N, Teede HJ. Acceptability of delivery modes for lifestyle advice in a large scale randomised controlled obesity prevention trial. BMC Public Health. 2015;15:699–713.View ArticlePubMedPubMed CentralGoogle Scholar
- Kozica S, Lombard C, Teede H, Ilic D, Murphy K, Harrison C. Initiating and continuing behaviour change within a weight gain prevention trial: a qualitative investigation. PLoS One. 2015;10(4):e0119773.View ArticlePubMedPubMed CentralGoogle Scholar
- Lombard C, Harrison C, Kozica S, Zoungas S, Ranasinha S, Teede H. Preventing weight gain in women in rural communities: a cluster randomised controlled trial. PLoS Med. 2016;13(1):e1001941.View ArticlePubMedPubMed CentralGoogle Scholar
- Kozica SL, Harrison CL, Teede HJ, Moran LJ, Ng N, Lombard CB. Engaging rural women in healthy lifestyle programs: insights from a randomized controlled trial. Trials. 2015. (In press).Google Scholar
- Australian Bureau of Statistics. Socio-economic indexes for areas: information paper, census of population and housing. In., vol. Canberra. Available: http://www.abs.gov.au/ausstats/abs@.nsf/mf/2033.0.55.001. Accessed 5 Jul 2013.
- Strauss AL, Corbin J. Basics of qualitative research, vol. 15. Newbury Park, CA: Sage publications; 1990.Google Scholar
- Davidson EJ. Evaluation methodology basics: the nuts and bolts of sound evaluation. London: Sage Publications; 2005.View ArticleGoogle Scholar
- Australian Bureau of Statistics. State and Regional Indicators, Victoria. Available: http://www.abs.gov.au. Accessed 5 May 2015.
- Harrison C, Teede H, Kozica S, Zoungas S, Lombard C. Individual, social and environmental factors and their association with weight in rural dwelling women. Aust N Z J Public Health. 2016. In press.Google Scholar
- Healthy together Victoria: Creating a healthier Victoria through a whole of systems approach. In. Available at: https://www2.health.vic.gov.au. Accessed 4 Sept 2014.
- Wang YC, McPherson K, Marsh T, Gortmaker SL, Brown M. Health and economic burden of the projected obesity trends in the USA and the UK. Lancet. 2011;378(9793):815–25.View ArticlePubMedGoogle Scholar
- Bonevski B, Randell M, Paul C, Chapman K, Twyman L, Bryant J, Brozek I, Hughes C. Reaching the hard-to-reach: a systematic review of strategies for improving health and medical research with socially disadvantaged groups. BMC Med Res Methodol. 2014;14(1):42.View ArticlePubMedPubMed CentralGoogle Scholar
- French SA, Gerlach AF, Mitchell NR, Hannan PJ, Welsh EM. Household obesity prevention: take action—a group‐randomized trial. Obesity. 2011;19(10):2082–8.View ArticlePubMedGoogle Scholar
- Han JC, Lawlor DA, Kimm S. Childhood obesity. Lancet. 2010;375(9727):1737–48.View ArticlePubMedPubMed CentralGoogle Scholar
- Skouteris H, McCabe M, Swinburn B, Newgreen V, Sacher P, Chadwick P. Parental influence and obesity prevention in pre-schoolers: a systematic review of interventions. Obes Rev. 2011;12(5):315–28.View ArticlePubMedGoogle Scholar
- Campbell K, Hesketh K, Crawford D, Salmon J, Ball K, McCallum Z. The Infant Feeding Activity and Nutrition Trial (INFANT) an early intervention to prevent childhood obesity: cluster-randomised controlled trial. BMC Public Health. 2008;8(1):103.View ArticlePubMedPubMed CentralGoogle Scholar
- Lutes LD, Winett RA, Barger SD, Wojcik JR, Herbert WG, Nickols-Richardson SM, Anderson ES. Small changes in nutrition and physical activity promote weight loss and maintenance: 3-month evidence from the ASPIRE randomized trial. Ann Behav Med. 2008;35(3):351–7.View ArticlePubMedGoogle Scholar
- Hutchesson M, Rollo M, Krukowski R, Ells L, Harvey J, Morgan P, Callister R, Plotnikoff R, Collins C. eHealth interventions for the prevention and treatment of overweight and obesity in adults: a systematic review with meta‐analysis. Obes Rev. 2015;16(5):376–92.View ArticlePubMedGoogle Scholar
- Klem ML, Viteri JE, Wing RR. Primary prevention of weight gain for women aged 25-34: the acceptability of treatment formats. Int J Obes. 2000;24(2):219–25.View ArticleGoogle Scholar
- Durlak JA, DuPre EP. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. 2008;41(3-4):327–50.View ArticlePubMedGoogle Scholar
- Hacker K, Tendulkar SA, Rideout C, Bhuiya N, Trinh-Shevrin C, Savage CP, Grullon M, Strelnick H, Leung C, DiGirolamo A. Community capacity building and sustainability: outcomes of community-based participatory research. Prog Community Health Partnersh. 2012;6(3):349–60.View ArticlePubMedPubMed CentralGoogle Scholar
- Huijg JM, Crone MR, Verheijden MW, van der Zouwe N, Middelkoop BJ, Gebhardt WA. Factors influencing the adoption, implementation, and continuation of physical activity interventions in primary health care: a Delphi study. BMC Fam Pract. 2013;14(1):142–50.View ArticlePubMedPubMed CentralGoogle Scholar
- Healthy Together Victoria. Healthy together healthy eating advisory service. In. Edited by Department of Health and Human Services SGoV. 2014. Available: http://heas.health.vic.gov.au/. Accessed 20 July 2015.