Evaluation of a large healthy lifestyle program: informing program implementation and scale-up in the prevention of obesity
Implementation Science volume 11, Article number: 151 (2016)
The Healthy Lifestyle Program for women (HeLP-her) is a low-intensity, self-management program which has demonstrated efficacy in preventing excess weight gain in women. However, little is known about the implementation, reach, and sustainability of low-intensity prevention programs in rural settings, where risk for obesity in women is higher than urban settings. We aimed to evaluate a low-intensity healthy lifestyle program delivered to women in a rural setting to inform development of effective community prevention programs.
A mixed method hybrid implementation and evaluation study, guided by the RE-AIM framework (addressing the Reach, Effectiveness, Adoption, Implementation, and Maintenance), was undertaken. Data collection tools included anthropometric measures, program checklists, questionnaires, and semi-structured interviews with participants and local stakeholders. The RE-AIM self-audit tool was applied to assess evaluation rigor.
Six hundred and forty-nine women from 41 relatively socio-economic disadvantaged communities in Australia participated: mean age 39.6 years (±SD 6.7) and body mass index of 28.8 kg/m2 (±SD 6.9). A between-group weight difference of −0.92 kg (95% CI −1.67 to −0.16) showed program effectiveness. Reach was broad across 41 towns with 62% of participants reporting influencing some of the health behaviors of their families. Strong implementation fidelity was achieved with good retention rates at 1 year (76%) and high participant satisfaction (82% of participants willing to recommend this program). Over 300 multi-level community partnerships were established supporting high adoption. Stakeholders reported potential capacity to implement and sustain the prevention program in resource poor rural settings, due to the low-intensity design and minimal resources required.
Our comprehensive RE-AIM evaluation demonstrates that an evidence-based obesity prevention program can be successfully implemented in real-world settings. The program achieved broad reach, effectiveness, and satisfaction at the community and stakeholder level, revealing potential for program sustainability. The evaluation addressed implementation knowledge gaps to support future obesity prevention program scale-up.
Treatment of established obesity via lifestyle interventions is challenging with poor engagement, high costs, and minimal sustainability [1, 2]. Weight loss is difficult to achieve at the individual level due to adaptive physiological responses post-weight loss, which almost universally drives weight regain [2, 3]. At the individual level, weight gain prevention is feasible, requiring only minor modifications to energy intake and expenditure . At the system level, primary prevention programs have the potential to reduce health care costs [1, 5]. In this context and with alarming escalation in obesity rates internationally, the World Health Organization (WHO) has identified prevention of excess weight gain as an international health priority .
Government prevention agencies have advocated the need for weight gain prevention programs in high-risk populations. Women have high rates of unhealthy weight gain [7, 8], and in many world regions, women have greater obesity prevalence than men  with higher subsequent obesity-related complications [10, 11]. Longitudinal data reveals that 20% of reproductive-aged women within a healthy weight range will become overweight within 5 years . For most individuals, weight gain is gradual over several decades and estimated at 600–800 g per year . Notably, even slight increases in weight of ~0.6 kg per year have been shown to increase the risk of breast cancer , hypertension , type 2 diabetes , and coronary heart disease  in women. Furthermore, as reproductive-aged women often have a key role in determining household food choices and sedentary behaviors, the need to invest in strategies to prevent excess weight gain in women is clear. In this context, we developed a low-intensity healthy lifestyle program for women, The Healthy Lifestyle Program for women (HeLP-her), shown to have efficacy for preventing weight gain in women in prior RCT’s in different settings and ethnically diverse populations [4, 17–19].
In developed countries, rural-dwelling women are more vulnerable than urban-dwelling women with lower socio-economic status and elevated rates of weight gain and obesity [20, 21]. Additional challenges include reduced access to primary health care services, resources, and trained health professionals . However, few healthy lifestyle programs have been implemented in rural settings [5, 23], and a systemic review has highlighted that the value of weight gain prevention programs in rural communities has not been established . As such, low-cost, low-intensity weight gain prevention programs are urgently needed in rural settings [20, 21].
Evaluation provides vital insights into how a program achieves efficacy and effectiveness. It also generates essential knowledge to drive implementation and scale-up [24–27] and deliver impact from research investment at the population level [28–32]. Despite this, few evaluations have been applied to weight gain prevention programs, leaving a major knowledge gap [24, 33, 34]. Common barriers to evaluation are lack of funding, time constraints, limited workforce knowledge, skills, and familiarity with evaluation methodologies and the dearth of valid evaluation tools .
The RE-AIM evaluation framework (Reach, Effectiveness, Adoption, Implementation, and Maintenance) explores program implementation and generalizability, focusing on the transferability of research findings into clinical practice and policy. The RE-AIM framework was developed specifically to evaluate health promotion interventions and encourages data collection at both the individual and organizational level [36, 37]. The RE-AIM framework has been applied numerous times to health promotion programs and to childhood obesity prevention programs [38, 39], highlighting its value and methodological rigor. However, on assessment of health promotion research programs and grants employing the RE-AIM framework, less than 10% had applied the complete RE-AIM reporting criteria [40, 41]. Most measured one or two RE-AIM dimensions only. The authors of the RE-AIM framework have recently developed criteria to assess the use of the RE-AIM framework .
The Healthy Lifestyle Program (HeLP-her) is an evidence-based weight gain prevention program for reproductive-aged women, which here was adapted for rural settings in the HeLP-her Rural trial, with an embedded hybrid implementation and evaluation design . This manuscript aims to (a) provide a summation of evaluation and implementation results from the large-scale HeLP-her Rural randomized controlled trial, utilizing the RE-AIM framework and (b) assess the application of the RE-AIM framework to the HeLP-her Rural program (utilizing the RE-AIM assessment criteria) , ultimately aiming to inform implementation and scale-up of obesity prevention programs broadly.
Program design, setting, theory, and implementation
The efficacy of the HeLP-her program has been established previously in two large RCT’s in a community setting for urban-dwelling women and in an antenatal clinic setting for pregnant women [4, 17]. Here, we have used an integrated community cluster RCT design to adapt and implement the HeLP-her program in relatively disadvantaged rural communities in the state of Victoria, Australia (HeLP-her Rural). As previously described, rural town selection was based on population size (2000–10,000 people) and distance from Melbourne central business district (CBD; towns located 100–400 km from CBD). Study randomization occurred at the town (cluster) level and analysis at the individual level. Overall, 41 rural townships met these criteria and were randomized to intervention or control towns. Randomization was conducted by the study biostatistician using a computer-generated randomization list. The primary outcome of HeLP-her Rural was the difference in weight gain between control and intervention groups at 12 months. Study methodology is published elsewhere .
The HeLP-her Rural program
In summary, the program involved a 1-year active intervention followed by a 1-year observation phase (yet to report). Control participants received a single general group health information session. The intervention aimed to improve participant’s self-management capacity through skill development including goal setting, problem solving, and relapse prevention underpinned by the self-determination theory  and motivational interviewing . Participants received the program via mixed delivery modes with minimal personal contact (one group session) and lifestyle advice delivered remotely (phone coaching, text messages, a program manual, and a website) (see Table 1 for further details). The study and embedded implementation-evaluation program were approved by the Monash Health Research Ethics Committee for research involving humans, and all participants provided written informed consent, project No. 12034B. Written consent was provided by all participants.
Program evaluation design and theory
This manuscript reports on the results of the embedded program evaluation using an effectiveness-implementation hybrid study design and involved the simultaneous testing of clinical outcomes and implementation strategies. This research design is thought to facilitate “more rapid translational gains, more effective implementation strategies and promote the collection of useful information for decision makers for scale-up” . This study design is most appropriate for use when effectiveness of the program of interest has been established previously; such is the case for the HeLP-her program. The HeLP-her Rural evaluation aimed to explore program implementation strategies in general within complex systems, assessing implementation rigor. Within this evaluation, process evaluation measures included program fidelity, recruitment strategies, dose of the program delivered and received, program acceptability, and contextual factors influencing program implementation. A summative evaluation investigated overall program effectiveness, quality, community outcomes, and potential for future scale-up. Results on program effectiveness (weight changes), implementation (dose delivered and received, fidelity, recruitment, and program context) and the potential for program sustainability at the organization level have been previously published [46–50]. However, these results are also summarized here to address all elements of the RE-AIM framework.
Program recruitment and implementation
Recruitment occurred from September 2012–April 2013 (Fig. 1) . Participant recruitment strategies were underpinned by a comprehensive communication and engagement plan and were deliberately simple and low cost to reflect community practice. To assist implementation, we focused on community integration within existing structures and engaged rural communities at multiple levels including local government departments, health services, primary schools, kindergartens, and community groups. Participants were recruited through the distribution of an invitation letter and flyer to women, and research staff visited each township to provide information in person to potential participants. All women aged 18–55 living in the 41 selected communities were invited by letter and community flyers to participate in this program. Program recruitment strategies are described elsewhere  (Table 1).
Data collection methods
The HeLP-her Rural evaluation used mixed method data collection design (Additional file 1) . These included the following: (1) sourcing and analyzing administration data, (2) checklists and log books completed by program leaders, (3) observations by research team, (4) anthropometric data, (5) questionnaires completed by participants, (6a) semi-structured interviews with a sub-group of participants, (6b) semi-structured interviews with rural stakeholders, and (7) an evaluation self-audit.
Administrative data and contextual data
To explore program reach and context, data from the Australian Bureau of Statistics (ABS) measuring Socio-Economic Indexes for Areas (SEIFA) of relative disadvantage was utilized . Potential scores ranged from 1 to 10 with a lower score indicating a greater level of social disadvantage relating to household total income, education attained, and unemployment rates.
Program checklists and log books
Program specific process evaluation checklists were developed in conjunction with the research team field notes to assess implementation fidelity, recruitment, dose delivered, and program context. Program log books were kept to monitor program communication with organizational program partners.
Observations by the research team
The intervention was delivered by a research team (dieticians and exercise physiologists), working in pairs, with one researcher delivering the intervention, the other observing implementation, collecting data, completing checklists and offering feedback to ensure delivery consistency. Program fidelity was addressed by all researchers undergoing a 1-day training workshop (August 2012) delivered by an experienced trainer (CB) and receiving ongoing support and utilizing program developed presentations and resources, promoting consistency of program delivery (Table 1).
Weight, height, and waist and hip circumference measurements were collected by the trained team at baseline, 12 months (end of active intervention), and 24 months (end of observation).
Questionnaires completed by participants
All participants completed program devised questionnaires at baseline (prior to intervention commencement) and 12 and 24 months including items on demographic characteristics, socio-cultural and physical environment, health status, and program satisfaction. Participant program satisfaction (overall program, information provided, program delivery, and support provided) was assessed on a program-devised Likert scale (1–5) with higher scores representing greater satisfaction.
Semi-structured (qualitative) participant and stakeholder interviews
In-depth semi-structured interviews were conducted with a sub-group of participant’s 6 months post intervention commencement. A criteria-based, convenience sampling approach as previously described [47, 48] involved women from ten communities (six intervention and four control). One trained researcher conducted all interviews (SK), guided by an interview schedule (Additional file 2). Forty-five participant interviews were conducted until data saturation was met, determined when no new ideas emerged from the interviews, as per standard methods .
Local stakeholders were identified as those who had a direct interest in the development, delivery, and outcomes of prevention programs and included local government employees, clinical health care providers (general practitioners, allied health, community nurses), and non-clinical health providers as previously described . These stakeholders had not directly implemented the HeLP-her Rural program, rather enabled access to local networks for the research team during program implementation. Stakeholders were asked to provide insight into the enablers, barriers, and strategies that would promote program implementation for weight gain prevention. Prior to interviews, each stakeholder was provided with an overview of the HeLP-her study background, the design, staffing, intensity, and implementation plan. Twenty-four semi-structured telephone interviews were conducted led by one researcher (SK) (Additional file 3).
Evaluation self-audit (meta-evaluation) of the HeLP-her Rural program
A meta-evaluation refers to the auditing and assessment of a program evaluation, determining if the evaluation has produced credible and justifiable conclusions . We applied a criteria-based tool to assess the application of the RE-AIM framework to the HeLP-her Rural evaluation, developed by Kessler et al. . This tool outlines a minimum set of “core” items (n = 31) producing total and item-specific scores for each of the five RE-AIM domains. This corresponds to a comprehensive application of the framework, termed “fully developed use of RE-AIM”. Items that are not applicable to the program were excluded from the total calculated score (Table 2) .
The statistical analysis plan has been previously reported as has statistical methods applied [42, 49]. Data analysis was conducted using STATA and SPSS version 19.0 for Windows. The trial was designed to have a statistical power of 80% to detect a difference of 1.0 kg in weight between groups at 1-year with the use of a two-sided test at a significance level of 0.05. Results are presented as mean (SD) for continuous and relative frequencies for categorical data.
Qualitative transcripts were analyzed thematically, with grounded theory principles of analysis enabling the identification, coding, and categorization of primary data patterns. All transcripts were independently analyzed and coded by two investigators, assisted by the NVivo Software program (QSR International Pty Ltd. Version 10, 2012, Victoria, Melbourne).
Q1: To what extent did the program reach the target group?
Broad program reach at both the community and organizational level was achieved. Groups that engaged readily included local government agencies, health workers (community health centers, medical clinics, and hospitals), community groups (women’s organizations, neighborhood houses, and sports clubs), education groups (primary schools, kindergartens, and child care centers), and private groups (local businesses and recreational centers) (Fig. 2). As previously reported, n = 649 women were recruited, representing ~10% of the eligible target population [47, 50]. Based on predefined and limited exclusion criteria, less than 12% of volunteers (n = 95) were excluded post screening (Fig. 1). This program reached townships of significant socio-economic disadvantage with 75% of townships having a SEIFA index of less than 4 (potential score range of 1–10 with lower scores indicating greater disadvantage) (Fig. 3).
The baseline age and BMI of participants was 39.6 ± 6.7 years and 28.8 ± 6.9 kg/m2, respectively. The participants had diverse education levels, household sizes, and income levels and were representative of the broader Victorian regional and rural population for women of a similar age (25–54 years) for income and education [50, 54, 55].
Q2. What were the key motivators and barriers to program engagement at the participant level (reach)?
As previously described in qualitative analysis, motivators for program participation were convenience of the program location and perceived program utility such as weight management and optimisation of lifestyle choices, as well as attending the program with peers. Barriers to engagement included lack of anonymity, self-consciousness, and segregated social networks in rural settings .
Q3: Are healthy lifestyles contagious amongst family members and the social networks of program participants?
Qualitative participant interviews highlighted that the HeLP-her Rural program prevention messages reached beyond participants to their families and households. Of the intervention participants interviewed, all reported influencing the health behaviors of their household to various degrees such as increasing water, fruit, and vegetables consumption and physical activity frequency and limiting high-fat snack foods and takeaways.
It’s a roll on effect because if you’re eating healthy yourself and doing healthy meals it follows on to the kids – [intervention participant]
Participants unable to influence their household health choices described barriers which primarily related to unsupportive or resistant family members.
(My husband) he’s my resistance band. He’s like, no, you can have Coke, you can have pies. It’s very hard (for me)- [control participant]
I’ve got two kids that are very fussy eaters…At the end of the day I can’t be bothered with the arguments and whinging - [control participant]
In contrast, women were less likely to influence the health behaviors of their social networks, citing personal and social reasons. Personal reasons related to feelings of “embarrassment,” wanting to maintain “privacy,” and “low confidence”.
My weight is something I’m very embarrassed about so I tend not to discuss it with friends. I don’t suppose my friends are struggling with it as I am - [control participant]
Social barriers to broader program reach included lack of socially acceptability to talk to friends about their weight and lifestyle choices as they do not want to be perceived as “self-absorbed” and “talking about myself all the time.” Other participants described that as many of their friends were already “very active” and “pretty healthy,” there was no need to discuss program learnings. Social isolation was another commonly described reason for not influencing social networks. Others noted beliefs that their friends were “unsupportive” and disinterested, “none of my friends are really into exercise and healthy lifestyles”.
Consistent with our qualitative findings, quantitative questionnaire data collected from intervention (n = 230) and control participants (n = 207) at 12 months showed that 62% of the participants reported influencing health behaviors of their families. However, there was no statistical significant difference between groups. The participants were most likely to influence their children (38%), friends (34%), and partners (30%).
Q4. Was the HeLP-her Rural program efficacious at preventing weight gain in intervention participants?
As previously reported, HeLP-her Rural prevented weight gain at 1 year with the mean weight change from baseline in the control group at +0.44 kg (95% CI −0.09 to 0.97) and the intervention group at −0.48 kg (95% CI −0.99 to 0.03), with a between-group difference of −0.92 kg (95% CI −1.67 to −0.16) or −0.88 kg (95% CI −1.62 to −0.13) adjusted for baseline values and clustering .
Q5. What were the enablers and barriers to behavior change sustainability within this program?
Behavior change continuation was facilitated by participant’s ability to apply the core program messages such as setting achievable behavior change goals, problem solving, and relapse prevention. Improved health knowledge, internal motivation, self-efficacy, and internal accountability all supported continued behavior change as previously described .
Q7. How many and which organizations supported HeLP-her Rural program implementation?
We partnered with more than 300 local organizations across the 41 rural communities with at least three partners from various community sectors in each community. Our partnering success rate was 95% amongst local stakeholders (Table 3). However, local partners were not required to deliver the intervention, rather to support program implementation and delivery. Local partners were highly valuable and reasons included the following: (1) providing local insights regarding community socio-cultural influences, (2) assisting program recruitment through promotion to professional and social networks, distribution of promotional material, and validation and legitimisation of the program, and (3) assisting program delivery (provision of venues to host the program). As local partners did not directly deliver the HeLP-her program, adoption by staff could not be directly assessed.
Q8. Do stakeholders value weight gain prevention programs?
According to stakeholders, prevention programs were highly valuable and aligned with rural organizational health priorities.
I would encourage my managers to support [prevention programs] because, the demand on the service for weight management is huge and it is particularly women - [Local dietician]
Stakeholders partnered with the HeLP-her Rural program team due to the underlying robust theory, evidence base, and simplistic low-intensity design.
The model (program theory) sounded like something worth promoting. That low commitment program… It sounded like a good program to support – [Health Service Manager].
In addition, many stakeholders valued the use of remote methods to deliver lifestyle advice, “It’s new, innovative ideas and I think that using digital medium is a great way.”
Indeed, due to the low intensity of the program and minimal resources required, many stakeholders discussed that their organization would likely have sufficient capacity to implement such a program.
I think we could probably deliver a program like that ourselves, with the correct information, I mean and tailoring it to what we’ve got. Actually I was sitting in on the program and thinking you know we could do this –[Community nurse]
Stakeholders provided recommendations to improve program delivery and optimize future scale-up of prevention programs. As previously reported, key recommendations included the development of multi-level partnerships, ongoing mentoring relationships via electronic communication, delivery of programs amongst outlying rural townships, and the provision of a suite of implementation resources to support cost and time-efficient implementation .
Q9. To what extent had the HeLP-her Rural program been implemented as per the study protocol?
Process evaluation revealed strong implementation fidelity and high dose delivered and received, confirmed through administration records, researchers observations, and completed program devised checklists, as previously reported. This highlighted the acceptability of low-intensity healthy lifestyle programs with mixed face-to-face and remote delivery modes in this population group. Qualitative participant interviews revealed that group education sessions were most valued, followed by text messages and phone coaching. Overall, delivery of lifestyle advice through multiple delivery modes (group sessions, phone coaching, text messages, and a program manual) was recommended by participants to optimize program acceptability and accommodate diverse learning styles . A full economic analysis is underway based on a pending 2-year data analysis and will be reported elsewhere.
Q10. Were participants satisfied with the HeLP-her Rural program experience?
Qualitative participant interviews reported a high level of program acceptability.
I think [the program] is fantastic and just like the fact that you held it in the first place, I thought oh wow, there’s people that are actually out there to help and yes, it felt like a privilege – [intervention participant]
The 1-year satisfaction surveys showed high participant satisfaction with 82% of the participants agreeing they would recommend this program, with more intervention participants recommending the program (90 versus 73%, P < 0.05) and being satisfied with program support than the controls (84 versus 61%, P < 0.05).
Q11. Is there evidence of an organizational demand for program continuation post study completion?
The HeLP-her program has been identified as an evidence-based community program by the Victorian State Government as part of a state-wide system-based approach to tackling the rising rates of obesity and preventing obesity-related conditions . This demonstrates the value of a self-management lifestyle program to prevent weight gain and the need for comprehensive evaluation and implementation evidence.
Q12. Utilizing a criteria-based meta-evaluation audit tool, does the HeLP-her Rural program measure all “core” RE-AIM dimensions?
Employing Kessler’s et al. RE-AIM meta-evaluation tool, this evaluation assessed all “core” elements of the RE-AIM framework, producing a score of 96% for “fully developed use across all RE-AIM dimensions” (Table 2) . We were unable to assess the domain “adoption at the researcher and staff level” as this was not applicable.
The HeLP-her, low-intensity healthy lifestyle program has now been delivered in a range of settings and populations in a series of RCT’s where it effectively improves lifestyle and prevents weight gain [4, 17, 49]. The HeLP-her Rural program achieved broad program reach across 41 rural townships with most participants reporting influencing some of the health behaviors of their families. Program implementation was supported by 300 multi-level partnerships with partners valuing program theory and low-intensity program design. Stakeholders reported capacity to locally implement and sustain the program. The HeLP-her Rural program prevented a weight gain of nearly 1 kg on average amongst women living in rural Australia. The US Agency for Healthcare Research and Quality defines a weight difference of 0.5 kg between groups as clinically significant and meaningful. Notably, a modeling study estimated that a 1-kg weight loss, if applied across the USA population, could avoid 2 million cases of diabetes, 1.5 million cases of cardiovascular disease, and more than 73,000 cases of cancer . These findings add new information on effective weight gain prevention strategies, mirroring current clinical guidelines and addressing international health priorities to halt the obesity epidemic.
The RE-AIM framework was developed to facilitate translation of research findings into improved population health outcomes, focusing on increasing the reporting of implementation strategies and external program validity . A systematic review of the RE-AIM framework shows broad application to health promotion programs mainly around chronic disease. However, the framework has been inconsistently and sometimes incorrectly applied across all five dimensions [40, 41]. Reporting all RE-AIM dimensions and criteria is critical with interdependent relationships important for public health impact . Updated RE-AIM criteria  recommend qualitative research to improve understanding of results, yet this is rarely employed [40, 41]. Here, we address key evaluation and obesity lifestyle prevention gaps by applying qualitative and quantitative research methods and extending evaluation to all RE-AIM dimensions. Subsequently, increasing the generalizability of our results to obesity prevention programs more broadly.
Here, we targeted rural women based on high rates of weight gain, risk of obesity and related complications, adverse impact on maternal and child short- and long-term health, and relative disadvantage for rural women. Despite reported difficulties in engaging socio-economic disadvantaged groups into research , we reached women from disadvantaged communities who were representative of regional women of a similar age (25–54 years) based on income, household size, and education . We note that participants reported directly influencing their household’s lifestyle behaviors, such as improving fruit and vegetable intake and physical activity frequency and reducing discretionary food and takeaway consumption. Research exploring the relationship between lifestyle changes in women participating in healthy lifestyle programs and family reach is limited. Further research exploring measurable impact of participant’s reports of influence with members of the household is needed. However, our results support the potential importance of targeting reproductive-aged women [59–61], due to their influential role in influencing household lifestyle behaviors and food choices .
The HeLP-her Rural program and evaluation has addressed the dearth of literature surrounding the acceptability of low-intensity weight gain prevention programs in young women [59, 63]. The program has successfully engaged 1127 women across three large RCT’s, emphasizing that low-intensity programs appeal to women from highly diverse socio-economic, ethnic, and educational backgrounds [4, 17, 47]. Furthermore, we have shown that a combination of delivery modes (face-to-face, phone coaching, text messages, and program manuals) optimizes program acceptability and delivers effectiveness. This is in keeping with a recent systematic review reporting the effectiveness of using diverse e-health methods (emailing, texting, phone apps, and websites) to prevent and manage obesity . Supporting the high value of weight gain prevention programs in women, 92% of intervention participants reported that they would recommend this program. Conversely, intensive weight loss programs are generally poorly received with low engagement and high attrition rates . The value and effectiveness of low-intensity weight gain prevention programs demonstrated here can inform the design and execution of prevention programs and strategies more broadly.
Implementing, scaling up, and sustaining evidence-based lifestyle programs in diverse real-world settings are notoriously challenging, yet they are fundamental to improve public health outcomes . Here, we report on the strong potential for program adoption and scale-up, related to broad community reach, effectiveness, low-intensity program design, and high acceptability at the participant and organizational levels. Supporting the value of the prevention-based programs at the organizational level, we successful partnered with local organizations. Stakeholders valued the underlying program theory and design and alignment of the program objectives with their organizational health priorities because this was a unique professional development opportunity. These partnerships were essential to optimizing program implementation, especially in relation to participant engagement. As previously reported , stakeholders recommended strategies for optimizing program sustainability and scale-up including building local capacity, developing partnerships and minimizing implementation costs [67, 68]. Moving forward, the HeLP-her program is ready for delivery by local rural health professionals with an interest in weight gain prevention. To date, training of rural communities by the HeLP-her Rural research team has occurred to support professional capacity building of the rural workforce. Funding has been provided by the Australian government preventative health taskforce to refine resources, engage, and adapt considering ethnic diversity and for initial steps in community roll-out . However, to enable program sustainability long-term funding is required, and partnerships with funders in a range of setting are currently in development.
Study strengths and limitations
Study strengths include utilizing a robust mixed methods evaluation design with an underpinning theoretical framework and a meta-evaluation that revealed all “core” RE-AIM elements were assessed. However, we note that “adoption” at the staff level was unable to be measured within this context due to program design. Limitations include that our program checklists exploring program fidelity and contextual influences were completed by researchers involved in the trial, rather than independent evaluators. Contextual factors and characteristics could also have been further explored. Furthermore, whilst comprehensive 2-year weight data and program economic analysis are underway, these data are needed to inform program maintenance. Given a large amount of data needed to meet all “core” RE-AIM dimensions, a staged approach to data collection and analysis in RE-AIM evaluation is common.
Evidence-based obesity prevention programs such as the HeLP-her healthy lifestyle program have potential for population level scale-up, with broad reach beyond direct participants, demonstrated effectiveness across a range of populations and settings, low-intensity design, and high acceptability at the community and stakeholder levels. Here, we report strong program implementation rigor and comprehensive application of the RE-AIM framework. Moving forward, there is a clear need to redefine lifestyle program success if we are to deliver public health impact. Success must extend beyond effectiveness to incorporate program reach, adoption and potential for implementation and sustainability in diverse real-world settings. This is critical to inform policy and scale-up of lifestyle programs to address the current obesity epidemic.
Lemmens VE, Oenema A, Klepp KI, Henriksen HB, Brug J. A systematic review of the evidence regarding efficacy of obesity prevention interventions among adults. Obes Rev. 2008;9(5):446–55.
Apovian CM, Aronne LJ, Bessesen DH, McDonnell ME, Murad MH, Pagotto U, Ryan DH, Still CD. Pharmacological management of obesity: an endocrine society clinical practice guideline. J Clin Endocrinol Metab. 2015;100(2):342–62.
Hinkle W, Cordell M, Leibel R, Rosenbaum M, Hirsch J. Effects of reduced weight maintenance and leptin repletion on functional connectivity of the hypothalamus in obese humans. PLoS One. 2013;8(3):e59114.
Lombard C, Deeks A, Jolley D, Ball K, Teede H. A low intensity, community based lifestyle programme to prevent weight gain in women with young children: cluster randomised controlled trial. BMJ. 2010;341:c3215.
Lombard CB, Deeks AA, Teede HJ. A systematic review of interventions aimed at the prevention of weight gain in adults. Public Health Nutr. 2009;12(11):2236–46.
World Health Organization. Obesity: preventing and managing the global epidemic. In: WHO Technical Report Series 894. Avaialble: http://www.who.int/nutrition/publications/obesity/WHO_TRS_894/en/. Accessed 3 Apr 2012.
Australian Government National Preventative Health Taskforce. Australia: the healthiest country by 2020. In. Avaialble: http://www.preventativehealth.org.au/internet/preventativehealth/publishing.nsf/Content/nphs-roadmap-toc. Accessed 28 Jun 2012.
Australian Longitudinal Study on Women’s Health. Australian women and their weight—a growing problem 2005 In. Available: http://www.alswh.org.au/. Accessed 4 Jan 2012.
Alwan A. Global status report on noncommunicable diseases 2010. In: World Health Organization. Available: http://www.who.int/nmh/publications/ncd_report2010/en/. Accessed 6 Mar 2015.
Guelinckx I, Devlieger R, Beckers K, Vansant G. Maternal obesity: pregnancy complications, gestational weight gain and nutrition. Obes Rev. 2008;9(2):140–50.
Hutchesson MJ, Hulst J, Collins CE. Weight management interventions targeting young women: a systematic review. J Acad Nutr Diet. 2013;113(6):795–802.
Andajani-Sutjahjo S, Ball K, Warren N, Inglis V, Crawford D. Perceived personal, social and environmental barriers to weight maintenance among young women: a community survey. Int J Behav Nutr Phys Act. 2004;1(1):15.
Trentham-Dietz A, Newcomb PA, Egan KM, Titus-Ernstoff L, Baron JA, Storer BE, Stampfer M, Willett WC. Weight change and risk of postmenopausal breast cancer (United States). Cancer Causes Control. 2000;11(6):533–42.
Huang Z, Willett WC, Manson JE, Rosner B, Stampfer MJ, Speizer FE, Colditz GA. Body weight, weight change, and risk for hypertension in women. Ann Intern Med. 1998;128(2):81–8.
Colditz GA, Willett WC, Rotnitzky A, Manson JE. Weight gain as a risk factor for clinical diabetes mellitus in women. Ann Intern Med. 1995;122(7):481–6.
Willett WC, Manson JE, Stampfer MJ, Colditz GA, Rosner B, Speizer FE, Hennekens CH. Weight, weight change, and coronary heart disease in women. Risk within the ‘normal’ weight range. JAMA. 1995;273(6):461–5.
Harrison CL, Lombard CB, Strauss BJ, Teede HJ. Optimizing healthy gestational weight gain in women at high risk of gestational diabetes: a randomized controlled trial. Obesity. 2013;21(5):904–9.
Lombard CB, Deeks AA, Ball K, Jolley D, Teede HJ. Weight, physical activity and dietary behavior change in young mothers: short term results of the HeLP-her cluster randomized controlled trial. Nutr J. 2009;8:17.
Harrison CL, Lombard CB, Teede HJ. Limiting postpartum weight retention through early antenatal intervention: the HeLP-her randomised controlled trial. Int J Behav Nutr Phys Act. 2014;11(1):1.
Janus ED, Laatikainen T, Dunbar JA, Kilkkinen A, Bunker SJ, Philpot B, Tideman PA, Tirimacco R, Heistaro S. Overweight, obesity and metabolic syndrome in rural southeastern Australia. Med J Aust. 2007;187(3):147–52.
Cleland V, Hume C, Crawford D, Timperio A, Hesketh K, Baur L, Welch N, Salmon J, Ball K. Urban-rural comparison of weight status among women and children living in socioeconomically disadvantaged neighbourhoods. Med J Aust. 2010;192(3):137–48.
Wakerman J, Humphreys JS, Wells R, Kuipers P, Entwistle P, Jones J. Primary health care delivery models in rural and remote Australia—a systematic review. BMC Health Serv Res. 2008;8(1):276.
Lemmens VEPP, Oenema A, Klepp K, Henriksen HB, Brug J. A systematic review of the evidence regarding efficacy of obesity prevention interventions among adults. Obes Rev. 2008;9(5):446–55.
Partridge S, Juan SH, McGeechan K, Bauman A, Allman-Farinelli M. Poor quality of external validity reporting limits generalizability of overweight and/or obesity lifestyle prevention interventions in young adults: a systematic review. Obes Rev. 2015;16(1):13–31.
Oldenburg B, Absetz P. Lost in translation: overcoming the barriers to global implementation and exchange of behavioral medicine evidence. Transl Behavioral Med. 2011;1(2):252–5.
Fixsen DL, Naoom SF, Blase KA, Friedman RM, Wallace F. Implementation Research: A Synthesis of the Literature. Tampa: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231); 2005.
Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE. Knowledge translation of research findings. Implement Sci. 2012;7(1):5060.
Baranowski T, Stables G. Process evaluations of the 5-a-day projects. Health Educ Behav. 2000;27(2):157–66.
Saunders RP, Evans MH, Joshi P. Developing a process-evaluation plan for assessing health promotion program implementation: a how-to guide. Health Promot Pract. 2005;6(2):134–47.
Grant A, Treweek S, Dreischulte T, Foy R, Guthrie B. Process evaluations for cluster-randomised trials of complex interventions: a proposed framework for design and reporting. Trials. 2013;14(1):15–25.
Burchett H, Umoquit M, Dobrow M. How do we know when research from one setting can be useful in another? A review of external validity, applicability and transferability frameworks. J Health Serv Res Policy. 2011;16(4):238–44.
Patel B, Patel A, Usherwood J, Stephen T, Harris M, Peiris D. A multifaceted quality improvement intervention for CVD risk management in Australian primary healthcare: a protocol for a process evaluation. Implement Sci. 2014;9:187–98.
Compernolle S, De Cocker K, Lakerveld J, Mackenbach J, Nijpels G, Oppert J-M, Rutter H, Texeira P, Cardon G, De Bourdeaudhuij I. A RE-AIM evaluation of evidence-based multi-level interventions to prevent obesity in adults: a systematic review. Int J Behav Nutr Phys Act (ISBNPA). 2014. (in press).
Kremers S, Reubsaet A, Martens M, Gerards S, Jonkers R, Candel M, de Weerdt I, de Vries N. Systematic prevention of overweight and obesity in adults: a qualitative and quantitative literature analysis. Obes Rev. 2010;11(5):371–9.
Kozica SL, Lombard CB, Harrison CL, Hider K, Teede HJ. Developing comprehensive health promotion evaluations: a methodological review. MedCrave Online J Public Health. 2015;2(1):1–7.
Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89(9):1322–7.
McKenzie R, Naccarella L, Stewart A, Thompson C. Targeting what matters in health promotion evaluation. Eval J Australasia. 2007;7(1):19–26.
Jenkinson KA, Naughton G, Benson AC. The GLAMA (Girls! Lead! Achieve! Mentor! Activate!) physical activity and peer leadership intervention pilot project: a process evaluation using the RE-AIM framework. BMC Public Health. 2012;12:55.
De Meij JS, Chinapaw MJ, Kremers SP, Van der Wal MF, Jurg ME, Van Mechelen W. Promoting physical activity in children: the stepwise development of the primary school-based JUMP-in intervention applying the RE-AIM evaluation framework. Br J Sports Med. 2010;44(12):879–87.
Kessler RS, Purcell EP, Glasgow RE, Klesges LM, Benkeser RM, Peek C. What does It mean to “employ” the RE-AIM model? Eval Health Prof. 2012;36(1):44–66.
Gaglio B, Shoup JA, Glasgow RE. The RE-AIM framework: a systematic review of use over time. Am J Public Health. 2013;103(6):e38–46.
Lombard C, Harrison C, Kozica S, Ng N, Teede H. Translation and implementation of an evidence based program to prevent weight gain in women. Obes Res Clin Pract. 2013;7:e76.
Deci EL, Ryan RM. Self determination. London: Wiley Online Library; 1985.
Rubak S, Sandbæk A, Lauritzen T, Christensen B. Motivational interviewing: a systematic review and meta-analysis. Br J Gen Pract. 2005;55(513):305–12.
Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012;50(3):217–26.
Kozica SL, Teede HJ, Harrison CL, Klein R, Lombard CB. Optimizing implementation of obesity prevention programs: a qualitative investigation within a large-scale randomized controlled trial. J Rural Health. 2015. (In press).
Kozica SL, Lombard CB, Harrison CL, Ilic D, Ng N, Teede HJ. Acceptability of delivery modes for lifestyle advice in a large scale randomised controlled obesity prevention trial. BMC Public Health. 2015;15:699–713.
Kozica S, Lombard C, Teede H, Ilic D, Murphy K, Harrison C. Initiating and continuing behaviour change within a weight gain prevention trial: a qualitative investigation. PLoS One. 2015;10(4):e0119773.
Lombard C, Harrison C, Kozica S, Zoungas S, Ranasinha S, Teede H. Preventing weight gain in women in rural communities: a cluster randomised controlled trial. PLoS Med. 2016;13(1):e1001941.
Kozica SL, Harrison CL, Teede HJ, Moran LJ, Ng N, Lombard CB. Engaging rural women in healthy lifestyle programs: insights from a randomized controlled trial. Trials. 2015. (In press).
Australian Bureau of Statistics. Socio-economic indexes for areas: information paper, census of population and housing. In., vol. Canberra. Available: http://www.abs.gov.au/ausstats/abs@.nsf/mf/2033.0.55.001. Accessed 5 Jul 2013.
Strauss AL, Corbin J. Basics of qualitative research, vol. 15. Newbury Park, CA: Sage publications; 1990.
Davidson EJ. Evaluation methodology basics: the nuts and bolts of sound evaluation. London: Sage Publications; 2005.
Australian Bureau of Statistics. State and Regional Indicators, Victoria. Available: http://www.abs.gov.au. Accessed 5 May 2015.
Harrison C, Teede H, Kozica S, Zoungas S, Lombard C. Individual, social and environmental factors and their association with weight in rural dwelling women. Aust N Z J Public Health. 2016. In press.
Healthy together Victoria: Creating a healthier Victoria through a whole of systems approach. In. Available at: https://www2.health.vic.gov.au. Accessed 4 Sept 2014.
Wang YC, McPherson K, Marsh T, Gortmaker SL, Brown M. Health and economic burden of the projected obesity trends in the USA and the UK. Lancet. 2011;378(9793):815–25.
Bonevski B, Randell M, Paul C, Chapman K, Twyman L, Bryant J, Brozek I, Hughes C. Reaching the hard-to-reach: a systematic review of strategies for improving health and medical research with socially disadvantaged groups. BMC Med Res Methodol. 2014;14(1):42.
French SA, Gerlach AF, Mitchell NR, Hannan PJ, Welsh EM. Household obesity prevention: take action—a group‐randomized trial. Obesity. 2011;19(10):2082–8.
Han JC, Lawlor DA, Kimm S. Childhood obesity. Lancet. 2010;375(9727):1737–48.
Skouteris H, McCabe M, Swinburn B, Newgreen V, Sacher P, Chadwick P. Parental influence and obesity prevention in pre-schoolers: a systematic review of interventions. Obes Rev. 2011;12(5):315–28.
Campbell K, Hesketh K, Crawford D, Salmon J, Ball K, McCallum Z. The Infant Feeding Activity and Nutrition Trial (INFANT) an early intervention to prevent childhood obesity: cluster-randomised controlled trial. BMC Public Health. 2008;8(1):103.
Lutes LD, Winett RA, Barger SD, Wojcik JR, Herbert WG, Nickols-Richardson SM, Anderson ES. Small changes in nutrition and physical activity promote weight loss and maintenance: 3-month evidence from the ASPIRE randomized trial. Ann Behav Med. 2008;35(3):351–7.
Hutchesson M, Rollo M, Krukowski R, Ells L, Harvey J, Morgan P, Callister R, Plotnikoff R, Collins C. eHealth interventions for the prevention and treatment of overweight and obesity in adults: a systematic review with meta‐analysis. Obes Rev. 2015;16(5):376–92.
Klem ML, Viteri JE, Wing RR. Primary prevention of weight gain for women aged 25-34: the acceptability of treatment formats. Int J Obes. 2000;24(2):219–25.
Durlak JA, DuPre EP. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. 2008;41(3-4):327–50.
Hacker K, Tendulkar SA, Rideout C, Bhuiya N, Trinh-Shevrin C, Savage CP, Grullon M, Strelnick H, Leung C, DiGirolamo A. Community capacity building and sustainability: outcomes of community-based participatory research. Prog Community Health Partnersh. 2012;6(3):349–60.
Huijg JM, Crone MR, Verheijden MW, van der Zouwe N, Middelkoop BJ, Gebhardt WA. Factors influencing the adoption, implementation, and continuation of physical activity interventions in primary health care: a Delphi study. BMC Fam Pract. 2013;14(1):142–50.
Healthy Together Victoria. Healthy together healthy eating advisory service. In. Edited by Department of Health and Human Services SGoV. 2014. Available: http://heas.health.vic.gov.au/. Accessed 20 July 2015.
The authors would like to thank all participants and stakeholders that gave up their time to participate in this study and provided valuable insights. We would also like to acknowledge Kim Hider for her assistance with designing the HeLP-her evaluation framework and Dr Lisa Moran for her input. Thank you to Dr. Nicole Ng, Rachel Miller, and Kelly Ashcroft for assistance with participant recruitment, data collection, and collation. Samantha Kozica is an NHMRC postgraduate scholar, Cheryce Harrison is a National Heart Foundation Postdoctoral Research Fellow (100168), and Helena Teede is an NHMRC Practitioner Fellow.
This trial was funded by a project grant from the National Health and Medical Research Council Australia (ID 1022951). The funders had no role in the study design, the data collection and analysis, the decision to publish, or the preparation of the manuscript.
Availability of data and materials
All HeLP-her data files are available from the Figshare database at the following link: http://dx.doi.org/10.6084/m9.figshare.1618679. Participant and stakeholder qualitative interview transcripts are available upon request.
CL and HT conceptualized the RCT. All authors provided intellectual input into the evaluation design and methodology. SK conducted the interviews, analyzed the interviews, and drafted the manuscript. All authors contributed to, reviewed, and approved the manuscript.
All authors declare that they have no competing interests.
Ethics approval and consent to participate
The study and embedded implementation-evaluation program were approved by the Monash Health Research Ethics Committee for research involving humans, and all participants provided written informed consent, project No. 12034B. Written consent was provided by all participants.
About this article
Cite this article
Kozica, S.L., Lombard, C.B., Harrison, C.L. et al. Evaluation of a large healthy lifestyle program: informing program implementation and scale-up in the prevention of obesity. Implementation Sci 11, 151 (2016). https://doi.org/10.1186/s13012-016-0521-4