- Open Access
Effectiveness of an intervention to facilitate the implementation of healthy eating and physical activity policies and practices in childcare services: a randomised controlled trial
Implementation Science volume 10, Article number: 147 (2015)
The primary aim of this study was to evaluate the effectiveness of an intervention to increase the implementation of healthy eating and physical activity policies and practices by centre-based childcare services. The study also sought to determine if the intervention was effective in improving child dietary intake and increasing child physical activity levels while attending childcare.
A parallel group, randomised controlled trial was conducted in a sample of 128 childcare services. Intervention strategies included provision of implementation support staff, securing executive support, staff training, consensus processes, academic detailing visits, tools and resources, performance monitoring and feedback and a communications strategy. The primary outcome of the trial was the proportion of services implementing all seven healthy eating and physical activity policies and practices targeted by the intervention. Outcome data were collected via telephone surveys with nominated supervisors and room leaders at baseline and immediately post-intervention. Secondary trial outcomes included the differences between groups in the number of serves consumed by children for each food group within the Australian Guide to Healthy Eating and in the proportion of children engaged in sedentary, walking or very active physical activity assessed via observation in a random subsample of 36 services at follow-up.
There was no significant difference between groups for the primary trial outcome (p = 0.44). Relative to the control group, a significantly larger proportion of intervention group services reported having a written nutrition and physical activity policy (p = 0.05) and providing adult-guided activities to develop fundamental movement skills (p = 0.01). There were no significant differences between groups at follow-up on measures of child dietary intake or physical activity.
The findings of the trial were equivocal. While there was no significant difference between groups for the primary trial outcome, the intervention did significantly increase the proportion of intervention group services implementing two of the seven healthy eating and physical activity policies and practices. High levels of implementation of a number of policies and practices at baseline, significant obesity prevention activity in the study region and higher than previously reported intra-class correlation of child behaviours may, in part, explain the trial findings.
Australian Clinical Trials Registry (reference ACTRN12612000927820).
Overweight and obesity in childhood increases the likelihood of adult obesity and its comorbidities, including cardiovascular disease, type-2 diabetes and certain cancers [1, 2]. Inadequate physical activity and poor nutrition are key risk factors for overweight and obesity, with such risk behaviours beginning to develop in the early years of a child’s life, prior to commencing formal schooling . As such, population-level interventions that aim to increase physical activity and improve diet quality during early childhood have been recommended [3–8].
Childcare services are an important setting for the delivery of obesity prevention interventions, given their potential to support population-level improvements in child diet and physical activity . In countries such as Australia, the United States and the United Kingdom, over half of all children aged 0 to 5 years spend a large proportion of their waking hours each day in centre-based childcare services [9–12]. Best practice guidelines [13, 14] as well as standards for licensing and accreditation  recommend that childcare services implement policies and practices known to improve the quality of children’s diet and increase the time children spend being physically active while in care. Specifically, such guidelines recommend that services develop centre-based nutrition and physical activity policies, implement guidelines for foods brought from home or provided by the service, provide structured fundamental movement skill activities, ensure staff role model healthy eating and physical activity behaviours, limit the provision of sweetened drinks and limit opportunities for screen time [13, 14, 16, 17]. Such recommendations are supported by empirical research which suggests that implementation of such policies and practices improve child diet and physical activity while in care and can prevent excessive weight gain [18–20].
Despite evidence to support the effectiveness of such guidelines, research suggests that their implementation by childcare services is substandard. Studies from the United States found that less than 60 % of childcare services had a written physical activity policy  with some services providing food which contributed less than 17 % of children’s recommended dietary intake for vegetables . Similarly, Australian research suggests that less than 50 % of childcare services have a written physical activity policy , only 46–60 % of services program time each day for fundamental movement skill development  and less than 5 % of childcare services provide adequate serves of vegetables, as recommended by national dietary guidelines . Further, 25 % of Australian childcare services provide daily opportunities for sedentary screen time , 48 % provide sweetened drinks  and 60 % allow children to bring lunchboxes containing more than one serve of high fat, salt or sugary foods or drinks .
The authors are aware of three randomised controlled trials that have been conducted with the aim of increasing the implementation of healthy eating and physical activity policies and practices in childcare services. The trials were conducted from 2008 to 2014 in the United States and Australia and involved between 17 and 82 childcare services [26–28]. Two of the trials sought to implement the United States Nutrition and Physical Activity Self-Assessment for Child Care program using a variety of implementation strategies including environmental self-assessment, selection of areas for change, educational workshops for childcare service staff and parents, targeted technical assistance, consultation visits and printed resources [26, 28]. Evaluated using an environmental observation score, both failed to significantly increase service implementation of nutrition and physical activity practices [26, 28], but one significantly improved the quantity and quality of service nutrition and physical activity policies . The third trial delivered an intervention comprising of professional development for childcare staff, resources and access to health promotion staff . Hardy and colleagues significantly increased the frequency of fundamental movement skill sessions, yet there were no between-group differences on five other measures of the physical activity environment nor was the intervention successful in achieving change to service food policies or in-service food activities .
While several frameworks exist to guide the development and implementation of interventions to more effectively increase the implementation of policies and practices in childcare (e.g. [29, 30]), few randomised trials have been conducted to assess the effectiveness of implementation strategies in this setting. Implementation research conducted in clinical settings has found that trials utilising multiple intervention strategies and guided by theory are more likely to be effective in changing practice . More extended periods of intervention support have been recommended to increase effectiveness . However, previous trials conducted in the childcare setting have either utilised a limited number of such strategies  trialled interventions without the guidance of theoretical frameworks [27, 28] or were delivered over a short period of time (less than 7 months) [26–28]. Further two of the three studies delivered interventions to only a small number of childcare services (15 services or less), limiting their capacity to guide approaches aimed at achieving population-level service improvements [27, 28].
The primary aim of the trial was to assess whether a multicomponent intervention, delivered over 12 months was effective in increasing the proportion of centre-based childcare services implementing healthy eating and physical activity policies and practices. The study also sought to determine if the intervention was effective in improving child dietary intake and increasing child physical activity levels while attending childcare and, as a potential adverse effect, if it increased the occurrence of injury among staff or children.
The trial was funded by the Australian National Preventive Health Agency (reference 95WOL2011) and was prospectively registered with the Australian New Zealand Clinical Trials Registry (reference ACTRN12612000927820). Ethical approval to conduct the study was obtained from the Hunter New England (reference 12/08/15/5.01) and the University of Newcastle (reference H-2012-0321) human research ethics committees. The research is reported in accordance with the requirements of CONSORT Statement .
Design and setting
A detailed protocol for the trial has been published elsewhere . A parallel group randomised controlled trial was conducted in 128 centre-based childcare services in the Hunter region of New South Wales, Australia, from August 2012 to July 2014.
Centre-based childcare services included pre-schools and long day-care services. Services in the region were ineligible if they: catered exclusively for children requiring specialist care (less than 1 % of services), provided all on-site meals to children (approximately 30 % of services) or were fully government funded (approximately 3 % of services), as the ethical clearance and intervention design were not appropriate for such services.
Nominated supervisors (service managers) at all eligible services were contacted by a research assistant and invited to provide consent: (1) for their service to participate in the study, (2) for their own participation in a computer-assisted telephone interview (CATI) survey and (3) for one of their room leaders (head teacher caring for 3 to 5 year-old children) to be contacted and invited to participate in a CATI survey. A subsample of nominated supervisors were randomly selected by a research assistant using a random number function and invited to provide consent for their service to participate in a 1-day, post-intervention observation to assess child dietary intake and physical activity behaviours.
Randomisation and allocation
Following the completion of baseline data collection, childcare services were randomly allocated to either the intervention or control condition by a research assistant using a random number function in a 1:1 (intervention: control) ratio. Services were not blind to study allocation.
Briefly, the 12 month intervention aimed to increase childcare service implementation of healthy eating and physical activity policies and practices. The policies and practices were developed based on best practice Australian healthy eating and physical activity guidelines for the childcare setting  and those shown to be associated with child healthy eating and physical activity behaviours [16, 17]. The healthy eating and physical activity policies and practices implemented by services included the following:
Development of written nutrition and physical activity policies
Staff monitoring of children's lunchboxes every day against written nutritional guidelines and provision of feedback to parents when a non-compliant food was packed
Provision of water or reduced fat milk (for children over the age of 2 years) only
Staff role modelling of physically active play and healthy eating every day
Staff provision of prompts and positive comments to children to encourage physical activity and healthy eating every day
Provision of adult-guided fundamental movement skill development activities every day for at least 75 % of children
Restriction of sedentary screen time to less than weekly.
The design of the intervention to support implementation of the policies and practices utilised Damschroder’s Consolidated Framework for Implementation Research . The Framework integrates 19 theoretical models and is composed of five major domains identified as influential in successful intervention implementation. The application of the relevant constructs to the intervention has been published as part of the study protocol . The intervention consisted of eight evidence-based strategies to facilitate childcare service implementation of the healthy eating and physical activity policies and practices [35–39]. The intervention strategies included the following:
Implementation support staff —The research team provided each service with a support staff member who provided on-going implementation support and positive reinforcement via face-to-face visits, telephone and email contact. Implementation support staff members had tertiary qualifications in nutrition and dietetics, health education and psychology and had previous experience in delivering similar initiatives in the childcare setting.
Securing executive support [41, 42] – Nominated supervisors were asked to lead the development and implementation of nutrition and physical activity policies, co-facilitate training workshops with implementation support staff and communicate expectations regarding the implementation of policies and practices to childcare service staff during staff meetings.
Provision of staff training —A series of three 1-h training workshops which focused on policy and practice implementation were provided on-site to childcare service staff and included both didactic and interactive components.
Employment of consensus processes [35, 44]— Following each staff training workshop, implementation support staff facilitated a discussion with nominated supervisors and childcare service staff to reach group agreement regarding an implementation strategy for the targeted policies and practices.
Provision of academic detailing visits [45, 46]—Following each staff training workshop, an academic detailing visit was conducted which involved support staff observing and providing immediate feedback to childcare service staff as they implemented the practices targeted by the intervention.
Provision of tools and resources  – All services received an electronic and hardcopy package of tools and resources to support childcare service staff to implement the healthy eating and physical activity policies and practices.
Performance monitoring and feedback [47, 48]—Verbal and written feedback describing service progress toward implementation of the targeted policies and practices was delivered at six intervals throughout the 12 month intervention, with feedback based on information collected via the baseline CATI, telephone contacts and face-to-face visits.
Employment of a communications strategy —Services received hard copy and electronic bimonthly newsletters which communicated key messages relating to the healthy eating and physical activity policies and practices. Services that implemented all policies and practices received a certificate of recognition, were acknowledged in newsletters and were used as case study examples.
Participating services randomised to the control group received three newsletters at the commencement, mid-point and conclusion of the 12 month intervention. The newsletters were provided in hard copy and electronic formats and contained information on healthy eating and physical activity unrelated to the specific policies and practices targeted by the intervention. Control group services did not receive any other intervention from the research team during the study period.
Data collection and measures
Surveys administered via CATI were conducted with the nominated supervisor and a room leader caring for children 3 to 5 years. Baseline data collection occurred between August and November 2012 and assessed childcare service characteristics and healthy eating and physical activity policies and practices. Follow-up CATI surveys were conducted immediately post-intervention between May and July 2014 and assessed healthy eating and physical activity policies and practices, staff and child injury, and in the intervention group, the acceptability of the intervention.
Nominated supervisors were asked to report on the following: service days and hours of operation, type of service (pre-school or long day-care service), postcode, number of enrolled and attending children, number of primary contact teaching staff and whether any Aboriginal and/or Torres Strait Islander children were enrolled. The items used to assess service characteristics have been used in other Australian surveys of childcare services conducted by the research team [22, 24, 50].
Primary trial outcome
Healthy eating and physical activity policy and practice implementation
The primary trial outcome was the difference over time between groups in the proportion of services implementing all seven healthy eating and physical activity policies and practices. The primary trial outcome represents service achievement of “best practice”, maximising the potential of the service to support child healthy eating and physical activity.
Both nominated supervisors and room leaders were asked to report on their service’s implementation of the seven healthy eating and physical activity policies and practices using items validated in a previous sample of 42 Australian childcare services . Nominated supervisors were asked to report on the implementation of whole-of-service policies and practices. Room leaders were asked to report on the implementation of specific healthy eating and physical activity policies and practices within their room. Each survey item and its respective percent agreement and Kappa value (K) are listed below in order to provide an indication of the level of agreement between nominated supervisor report and independent observation .
Presence of written nutrition (75 %, K = 0.50) and physical activity policies (79 %, K = 0.59)
Staff monitoring of children’s lunchboxes against written nutritional guidelines (84 %, K = 0.69) and provision of feedback to parents when a non-compliant food is packed (68 %, K = 0.34)
Provision of water (89 %, K = 0.78) or reduced fat milk only (79 %, K = 0.57) to children
Staff role modelling of physically active play (69 %, K = 0.39) and healthy eating (94 %, K = 0.89) every day
Staff provision of prompts and positive comments to children to encourage physical activity (80 %, K = 0.60) and healthy eating (86 %, K = 0.71) every day
Provision of adult-guided fundamental movement skill development activities (53 %, K = 0.06) every day to at least 75 % of children (60 %, K = 0.20)
Restriction of sedentary screen time (58 %, K = 0.17) to less than weekly.
Secondary trial outcomes
In order to assess if the hypothesised improvements in implementation of the healthy eating and physical activity policies and practices was sufficient to yield improvements to child diet and physical activity while attending childcare, observations of child dietary intake and physical activity levels were undertaken. The 1-day observation was conducted during core service hours (9 am–3 pm) in a random subsample of intervention and control group childcare services at follow-up. One of four trained observers attended each service to observe both child dietary intake and physical activity during the 1-day observation. Observers did not participate in the delivery of the intervention and were blind to service group allocation.
Child dietary intake
Secondary trial outcomes included the differences between groups at follow-up in the mean number of serves consumed by children for each food group within the Australian Guide to Healthy Eating (vegetables, fruit, grains, meat and meat alternatives, milk, yoghurt and cheese and discretionary foods). Child dietary intake was assessed during the 1-day observation using a modified version of the Dietary Observation for Child Care protocol . The Dietary Observation for Child Care is a validated method for recording child-level dietary intake in 2 to 5 year-olds  and has been used extensively in the childcare setting [21, 53, 54]. Dietary intake was assessed in three children per service by an observer who visually estimated and recorded all types and portions of foods and drinks provided to and consumed by the children, along with amounts remaining after finishing a meal or snack . This was recorded for every food or drink item supplied by parents in the child’s lunchbox and offered to the child during the observation period. The children were randomly selected by asking the room leader at each service to identify the three children with the most recent birthdays. Following the completion of the observation, the numbers of serves for each of the Australian Guide to Healthy Eating food groups was generated by a qualified dietitian. The number of serves consumed for each food group was calculated using the weight of the food according to a nutrient database  and the standard serve size of the food according to the Australian Guide to Healthy Eating . Discretionary foods were classified using the Australian Guide to Healthy Eating with reference to the Australian Bureau of Statistics Discretionary Food List where unclear .
Observers were trained according to the Dietary Observation for Child Care protocol . Prior to undertaking the observations, the observers completed a 20-food certification test. The observer results were compared to the actual measured amounts of foods and a tolerance level was set for each of the 20 items. The observers correctly described more than 90 % of items within the test and reached between 75 and 100 % agreement with actual measured amounts for the 20 food and drink items.
Child physical activity
Secondary trial outcomes included the differences between groups at follow-up in the proportion of children engaged in sedentary, walking or very active physical activity during all observations, structured physical activity and outdoor free play sessions. Child physical activity levels were assessed at the same 1-day observation by the same observer, using a modified version of the System for Observing Play and Leisure in Youth (SOPLAY) tool and protocol . SOPLAY is a standardised instrument for assessing physical activity levels in recreational settings using systematic, momentary time sampling of a predetermined area . SOPLAY has been found to be both valid and reliable in school-aged children  and has been previously used to assess physical activity in the childcare setting . The observer coded all structured physical activity and outdoor free play sessions that occurred between 9 am and 3 pm at each service. Prior to the commencement of each physical activity session, observers recorded key aspects of the physical environment including location (inside or outside), type of session (structured physical activity or free play), scan start time and any equipment available for use. During each scan, the observers assessed the level of child physical activity by counting the number of children engaged in sedentary, walking or very active physical activity in 10-min intervals for the duration of each session.
Observers were trained according to the standardised SOPLAY protocol . The SOPLAY assessment DVD was used to assess each observer’s ability to independently scan and code physical activity levels quickly and accurately. Of the 28 video clips in the assessment, observers must have correctly counted the number of people engaged in either sedentary, walking or very active activity in each clip to receive one point. Scores ranged between 61 and 71 %.
Adverse effects—staff and child injury
Given an increase in child physical activity levels could potentially increase the risk of child injury , nominated supervisors in both the intervention and control groups were asked to report on the number of staff and children involved in adverse events in their service. Adverse events were defined as injuries requiring documentation during the previous 12 months.
Acceptability of the intervention
Nominated supervisors and room leaders in the intervention group were asked to respond on a four-point Likert scale (strongly agree, agree, disagree, strongly disagree) to a series of statements assessing the acceptability of the intervention resources, training and support provided to services.
Delivery of the intervention
The delivery of each of the eight intervention strategies was assessed by an independent research assistant using project records maintained by each implementation support staff member.
Blinding of CATI interviewers
CATI interviewers did not participate in the delivery of the intervention and were blind to service group allocation. To assess whether blinding was maintained, after collection of follow-up data, interviewers were asked to nominate the group to which they thought the service had been allocated.
For descriptive purposes and to aid an assessment of any external influences on the trial findings, a systematic search was conducted to describe the context in which the trial was conducted [62, 63]. Local news archives, websites of national and New South Wales health and education departments, accreditation standards and national healthy eating and physical activity guidelines were reviewed to identify the existence of or changes in government policy, standards, funded programs, or guidelines that may influence the healthy eating and physical activity environments of childcare services. The search included the 12 months prior to and during the 12 month intervention.
Sample size calculations
Primary trial outcome
Based on previous research, a 20 % study attrition rate of services was anticipated . Given this, recruitment of 128 services into the trial at baseline would be sufficient to provide follow-up data from approximately 102 childcare services (51 per group) and enable the detection of an absolute difference between groups in the proportion of services implementing all policies and practices of 27 with 80 % power and an alpha value of 0.05. This was based on an expected prevalence of control group services implementing all policies and practices at follow-up of 25 %.
Secondary trial outcomes
Assuming a consent rate of 80 %, inviting a random subsample of 42 services to participate in the post-intervention observations would be sufficient to provide data from approximately 34 childcare services (17 services per group). This would enable the detection of an absolute difference between groups in very active physical activity of 4.3 % with 80 % power, an alpha of 0.05 and based on an intra-class correlation coefficient (ICC) of 0.02. This was based on estimations of four physical activity sessions per service, four 10-min scans per session and 20 children per 10-min scan. This sample was also sufficient to detect an absolute difference between groups in the mean number of serves for each food group of 0.3 serves with 80 % power, an alpha of 0.05 and based on an ICC of 0.02. This was based on estimations of three children being observed at each service (51 children per group) and a standard deviation of 0.5.
All statistical analyses were performed using SAS (version 9.3) statistical software. All statistical tests were two tailed with an alpha value of 0.05.
Descriptive statistics were used to describe the service characteristics of intervention and control group services at baseline. Socioeconomic characteristics were determined using service postcodes, which were classified as being in the top or bottom 50 % of New South Wales according to the Socio-economic Indices for Areas (SEIFA). Geographic characteristics of the service locality were classified as either urban or rural according to the Australian Statistical Geography Standard.
Healthy eating and physical activity policy and practice implementation
The primary trial outcome was analysed under an intention-to-treat framework using all available data. A logistic regression model was developed to determine group-by-time changes in the proportion of services implementing all healthy eating and physical activity policies and practices from baseline to follow-up. The logistic regression model included terms for time, group (intervention or control) and group-by-time interaction. A sensitivity analysis was performed by imputing baseline observations at follow-up for missing data. The same method of analysis (using six separate logistic regression models) was used to assess group-by-time changes in the following subgroups: service type (pre-school or long day-care service), socioeconomic characteristics (top or bottom 50 % of New South Wales) and geographic characteristics (urban or rural). As the study was not powered to test any hypotheses relating to such subgroups, these results are provided for descriptive purposes only. The following post hoc exploratory analyses were also performed: first, separate logistic regression models were used to determine group-by-time changes in the proportion of services implementing each of the individual policies and practices from baseline to follow-up. Second, a linear regression model was used to assess whether there was a significant difference over time between groups in the mean number of policies and practices implemented.
Child dietary intake
The amount of food consumed by each child was calculated using the food consumption equation, defined as: amount served less (amount remaining ± amount wasted or added) . Descriptive statistics were used to assess child dietary intake data according to each of the Australian Guide to Healthy Eating food groups. A linear regression model was used to assess whether there was a significant difference between groups at follow-up in the mean number of serves for each food group (vegetables; fruit; grains; meat and meat alternatives; milk, yoghurt and cheese and discretionary foods). The model was adjusted for potential clustering effect.
Child physical activity
Descriptive statistics were used to assess the proportion of observations of the children’s physical activity levels. A logistic regression model was developed to assess whether there was a significant difference between groups at follow-up in the proportion of children engaged in sedentary, walking or very active physical activity. A generalised estimating equation (GEE) framework was utilised to account for potential clustering effects of the service (level one) and the SOPLAY session (level two). Analyses were performed on all observations, as well as on subgroups of the data including the type of physical activity (structured physical activity or outdoor free play session).
Acceptability of the intervention
Descriptive statistics were used to assess the delivery and acceptability of the intervention. Acceptability data was calculated using the percentage of nominated supervisors and room leaders that reported either “strongly agree” or “agree” to each item.
Of the 253 childcare services in the study region, 128 (65 %) nominated supervisors consented for their service to participate in the study. Of these, 120 services (95 %) provided follow-up data (Fig. 1). The baseline characteristics of intervention and control group services that completed the CATI survey at both baseline and follow-up are described in Table 1. There were no differences between the characteristics of services that provided follow-up data and those that did not (p = 0.22–1.00). A randomly selected subsample of 42 nominated supervisors were invited to participate in a 1-day observation at follow-up, with 36 (86 %) consenting. There were no differences between groups in the baseline characteristics of services that did and did not consent to participate in the observations.
Primary trial outcome
Healthy eating and physical activity policy and practice implementation
There was no significant difference over time between groups in the proportion of services implementing all healthy eating and physical activity policies and practices, the primary trial outcome (p = 0.44) (Table 2). Relative to the control group, a significantly larger proportion of intervention group services reported having a written nutrition and physical activity policy (p = 0.05) and providing adult-guided fundamental movement skill development activities (p = 0.01) (Table 3). There was a significant difference between groups in the mean number of healthy eating and physical activity policies and practices implemented over time, favouring the intervention group (p = 0.05). Data from the subsample of services attended by observers at follow-up found significant improvement in the service implementation of staff role modelling of physically active play and healthy eating (p = 0.05) but not for other policies or practices (p = 0.27–0.96). There were no significant differences between the characteristics of services that were implementing all healthy eating and physical activity policies and practices at baseline and those that were not (p = 0.09–0.87).
Secondary trial outcomes
Child dietary intake
There were no significant differences between groups at follow-up in the mean number of serves consumed by children for each food group (p = 0.14–0.96) (Table 4).
Child physical activity
There were no significant differences between groups at follow-up in the proportion of children engaged in sedentary, walking or very active physical activity during all observations (p = 0.54), structured physical activity (p = 0.64) and outdoor free play sessions (p = 0.37) (Table 4).
Adverse effects—staff and child injury
At follow-up in the intervention group, the mean number of staff injuries during the previous 12 months was 0.77 (confidence interval (CI) 0.49–1.06) and mean number of serious child injuries was 0.72 (CI 0.39–1.05). In the control group, the mean number of staff injuries during the previous 12 months was 0.84 (CI 0.42–1.26) and mean number of serious child injuries was 0.90 (CI 0.52–1.29). There was no significant difference between groups in the mean number of staff (p = 0.80) or child (p = 0.47) injuries during the previous 12 months.
Acceptability of the intervention
All nominated supervisors, and 98 % of room leaders, found the implementation support to be beneficial to their service (Table 5). Ninety-five percent of nominated supervisors and room leaders stated that ongoing implementation support would be useful, and just four nominated supervisors and three room leaders would have preferred less support throughout the 12-month intervention.
Delivery of the intervention
Table 6 shows the proportion of childcare services in the intervention group that received each of the intervention strategies. All services were offered and accepted 12 months of implementation support via telephone contact from an implementation support staff member. Implementation support was staggered based on staffing availability from January 2013 to April 2014. Ninety-four percent of nominated supervisors demonstrated executive support for the trial via co-facilitation of training workshops with implementation support staff and participation in consensus processes. Seventy-seven percent of services received all three staff training workshops and 76 % received all three academic detailing sessions. A total of 69 % of services received the full complement of all eight intervention strategies.
Blinding of CATI interviewers
At follow-up, interviewers correctly identified the services' group allocation in 70 % (p = <0.001) of nominated supervisor surveys and 57 % (p = 0.68) of room leader surveys.
Throughout the 12 months prior to, and during the 12 month intervention, two government guidelines that may have influenced the healthy eating and physical activity environments of childcare services were introduced. First, the Australian Dietary Guidelines (including the Australian Guide to Healthy Eating) were revised and released in February 2013 . Second, the National Accreditation Standards for the childcare sector (The National Quality Framework) came into effect in the 12 months prior to intervention delivery (January 2012) . In addition to government guidelines, two government-sponsored funded programs were identified. The first, a state-based program known as “Munch & Move”, was implemented during the study period . Project records show that 80 % of intervention group services and 12 % of control group services attended training in healthy eating and physical activity provided by the “Munch & Move” program during the study period. Secondly, Australia’s largest community-based obesity prevention program, known as “Good for Kids. Good for Life” was conducted in the study region from 2006 to 2011 . Project records provided by the program show 45 % of intervention group services and 52 % of control group services attended training in healthy eating and physical activity provided by the “Good for Kids. Good for Life” program during the period from 2006 to 2011.
Internationally, this is one of a small number of randomised controlled trials conducted to test an intervention aimed at increasing healthy eating and physical activity policy and practice implementation in childcare services. The study did not find a significant intervention effect on the primary trial outcome, the proportion of intervention group services implementing all seven policies and practices. However, the intervention significantly increased the proportion of intervention group services implementing two of these policies and practices (written nutrition and physical activity policy and provision of adult-guided activities to develop fundamental movement skills) and the mean number of policies and practices implemented.
The study findings are similar to that of the previously conducted randomised controlled trials in the childcare setting. Ward and colleagues found a non-significant improvement of 11 % in service nutrition and physical activity environments, policies and practices; Hardy and colleagues demonstrated a significant improvement in the provision of one of the six physical activity practices targeted by the intervention (frequency of fundamental movement skills sessions per week increased from 1.3 to 3.2), and Alkon and colleagues found significant improvements in the quantity of service policies (mean policy score increased from 0.89 to 5.17 (nutrition) and 0 to 2.82 (physical activity)) but no improvement in any of the 14 nutrition and physical activity practices targeted by the intervention [26–28]. It was anticipated that the current trial would yield an improvement of at least 20 % in the primary trial outcome, given the substantial increase in the number of intervention components, the duration of intervention support relative to past interventions, and evidence of similar effects sizes for implementation interventions of similar intensities in other settings .
There are several factors that may have limited the effectiveness of the intervention on the primary trial outcome. First, the primary outcome was a composite measure, requiring implementation of all seven targeted policies and practices by services. However, five of the seven policies and practices were being implemented by 80 % or more of intervention group services at baseline, limiting scope for further improvements. Second, the trial did not exclude services who were already implementing all policies and practices at baseline (24 % of intervention services). For these services, the benefit of further intervention is likely to be minimal. Third, at follow-up, the proportion of control services implementing all policies and practices increased by 17 %. Such substantial improvement in implementation in control groups has not been reported by other trials [26–28]. Context evaluation suggests that co-intervention in the control group and other important contextual factors such as the introduction of national dietary guidelines and the National Accreditation Standards occurring at the time of the trial may have facilitated policy and practice implementation in the control group, reducing the likelihood of an intervention effect. Finally, some policies and practices were particularly difficult for some services to implement. Anecdotally, for example, implementation support staff reported that most services that were not providing feedback to parents when non-compliant foods were packed found this practice to be particularly challenging, citing concerns about adverse reactions from parents.
In the context of the limited impact of the intervention on implementation of policies and practices, it is perhaps unsurprising that the intervention did not yield significant improvements in child dietary intake or physical activity levels in care. However, while non-significant, the effect size achieved for child very active physical activity (>4 %) and fruit intake (>1/3 of a serve) was consistent with the effect size on which the study had been powered to detect a priori. However, the ICC found in this study was far higher (0.34 for physical activity and 0.11 for fruit) than that which was predicted for the study (0.02) on which sample size calculations were based. The high ICC substantially reduced the effective sample size of the current study. Future studies using similar observational procedures to assess child diet and physical activity may require larger samples in order to detect clinically meaningful effects.
Strengths of the study include the trial’s randomised design, delivery of the intervention to a large population of childcare services, high study retention and the inclusion of child behavioural measurements at follow-up. The inclusion of a comprehensive set of process measures also provided a rich source of information to interpret the study findings. However, a number of study limitations are worth noting. First, the study relied on the self-report of nominated supervisors and room leaders for the measurement of service policies and practices, which may have introduced biases such as social desirability bias. While the survey items have been validated in a sample of Australian childcare services, a number of practices had only slight, fair or moderate agreement (kappa <0.6) . Future studies should look to conduct a more rigorous assessment of implementation, such as direct observation of service policies and practices. Second, the trial did not measure change in perceived barriers and enablers to implementing the policies and practices and, as such, any mechanisms that may have facilitated the outcome of the intervention could not be investigated. Third, the intervention was multi-component and the effectiveness of the individual intervention strategies was unable to be determined. Future studies could examine the effectiveness of individual intervention strategies. Fourth, process evaluation did not include an assessment of the timing of when childcare services implemented each policy or practice. As such, the exposure of individual children to each policy and practice is unknown, preventing any assessment of the impact of such exposure on child healthy eating and physical activity. Fifth, the study assessed child dietary intake and physical activity levels on one day at follow-up. Repeated dietary intake and physical activity observations conducted over multiple days may provide a more robust measure of these behaviours during attendance at childcare. Sixth, the study did not include services that provide on-site meals to children. Future research may consider evaluating the impact of an intervention targeting food-provision practices specific to such childcare services.
Despite the limitations, the study represents an important contribution to the limited literature regarding the implementation of obesity prevention interventions in the childcare setting. The findings demonstrate that, among a group of services where policy and practice implementation was generally high at baseline, the investment of significant implementation support to achieve “best practice” implementation may not yield significant improvements in the proportion of services achieving this goal. Investment in implementation support, therefore, may be better directed at services where policies and practice implementation is poorer initially, and where they may be greater scope for improvement. Future research to test the effectiveness of the intervention on such services is warranted. Given the high acceptability of the intervention strategies, prospective interventions may consider utilising strategies that best address the reported barriers to policy and practice implementation. For example, the addition of intervention strategies to garner the support for healthy eating and physical activity policies and practices by parents and carers may improve the effectiveness of future interventions, given their influence on service operation and activities . Finally, the intervention did improve some policies and practices, including the implementation of adult-guided activities to develop fundamental movement skills in children. An improvement in the implementation of fundamental movement skills activities was also noted in a randomised trial by Hardy and colleagues . Common to both interventions was staff training, resources and the use of implementation support staff. Such findings suggest that these intervention strategies may be particularly effective in supporting the implementation of this practice and should be retained in future interventions.
World Cancer Research Fund/American Institute for Cancer Research. Food, nutrition, physical activity, and the prevention of cancer: a global perspective. 2007. http://www.dietandcancerreport.org/cancer_resource_center/downloads/Second_Expert_Report_full.pdf. Accessed 30 Jan 2015.
Ezzati M, Lopez A, Rodgers A, Murray C. Comparative quantification of health risks: global and regional burden of disease attributable to selected major risk factors. World Health Organization. 2004. http://www.who.int/healthinfo/global_burden_disease/cra/en/. Accessed 30 Jan 2015.
Birch LL, Ventura AK. Preventing childhood obesity: what works? Int J Obes [London]. 2009;33:S74–81.
Dietz WH. Critical periods in childhood for the development of obesity. Am J Clin Nutr. 1994;59(5):955–9.
Hesketh KD, Campbell KJ. Interventions to prevent obesity in 0–5 year olds: an updated systematic review of the literature. Obesity. 2010;18(S1):S27–35.
Goldfield GS, Harvey A, Grattan K, Adamo KB. Physical activity promotion in the preschool years: a critical period to intervene. Int J Environ Res Public Health. 2012;9(4):1326–42.
Swinburn B, Gill T, Kumanyika S. Obesity prevention: a proposed framework for translating evidence into action. Obes Rev. 2005;6(1):23–33.
World Health Organization. Population-based approaches to childhood obesity prevention. World Health Organisation. 2012. http://apps.who.int/iris/bitstream/10665/80149/1/9789241504782_eng.pdf?ua=1. Accessed 30 Jan 2015.
Story M, Kaphingst KM, French S. The role of child care settings in obesity prevention. Future Child. 2006;16(1):143–68.
Mamedova S, Redford J. Early childhood program participation, from the National Household Education Surveys Program of 2012 [NCES 2013-029.REV]. National Center for Education Statistics, Institute of Education Sciences, US Department of Education. 2015. http://nces.ed.gov/pubs2013/2013029rev.pdf. Accessed 30 Jan 2015.
Huskinson T, Pye J, Medien K, Dobie S, Ferguson C, Gardner C, et al. Childcare and early years survey of parents 2011. 2013. https://www.gov.uk/government/statistics/childcare-and-early-years-survey-of-parents-2011. Accessed 30 Jan 2015.
Australian Bureau of Statistics. Experimental estimates of preschool education [No. 4240.0]. Australian Bureau of Statistics. 2011. http://www.abs.gov.au/ausstats/abs@.nsf/Lookup/4240.0main+features12011. Accessed 30 Jan 2015.
Commonwealth of Australia. Get up and grow healthy eating and physical activity for early childhood. 2013. http://www.health.gov.au/internet/main/publishing.nsf/Content/phd-early-childhood-nutrition-resources. Accessed 30 Jan 2015.
McWilliams C, Ball SC, Benjamin SE, Hales D, Vaughn A, Ward DS. Best-practice guidelines for physical activity at child care. Pediatrics. 2009;124(6):1650–9.
Australian Children's Education and Care Quality Authority. National quality framework. 2012. http://www.acecqa.gov.au/national-quality-framework. Accessed 30 Jan 2015.
Larson N, Ward D, Neelon S, Story M. What role can child-care settings play in obesity prevention? A review of the evidence and call for research efforts. J Am Diet Assoc. 2011;111(9):1343–62.
McPherson ME, Homer CJ. Policies to support obesity prevention for children: a focus on of early childhood policies. Pediatr Clin N Am. 2011;58(6):1521–41.
Trost SG, Fees B, Dzewaltowski D. Feasibility and efficacy of a “move and learn” physical activity curriculum in preschool children. J Phys Act Health. 2008;5(1):88.
de Silva-Sanigorski AM, Bell AC, Kremer P, Nichols M, Crellin M, Smith M, et al. Reducing obesity in early childhood: results from Romp & Chomp, an Australian community-wide intervention program. Am J Clin Nutr. 2010;91(4):831–40.
Adams J, Zask A, Dietrich U. Tooty fruity vegie in preschools: an obesity prevention intervention in preschools targeting children’s movement skills and eating behaviours. Health Promot J Austr. 2009;20(2):112–9.
Ball S, Benjamin S, Ward D. Dietary intakes in North Carolina child-care centers: are children meeting current recommendations? J Am Diet Assoc. 2008;108(4):718–21.
Wolfenden L, Neve M, Farrell L, Lecathelinais C, Bell C, Milat A, et al. Physical activity policies and practices of childcare centres in Australia. J Paediatr Child Health. 2011;47(3):73–6.
Yoong S, Skelton E, Jones J, Wolfenden L. Do childcare services provide foods in line with the 2013 Australian Dietary guidelines? A cross-sectional study. Aust N Z J Public Health. 2014;38:595–6.
Bell AC, Davies L, Finch M, Wolfenden L, Francis JL, Sutherland R, et al. An implementation intervention to encourage healthy eating in centre-based child-care services: impact of the Good for Kids Good for Life programme. Public Health Nutr. 2014;9:1–10.
Kelly B, Hardy LL, Howlett S, King L, Farrell L, Hattersley L. Opening up Australian preschoolers’ lunchboxes. Aust N Z J Public Health. 2010;34(3):288–92.
Ward DS, Benjamin SE, Ammerman AS, Ball SC, Neelon BH, Bangdiwala SI. Nutrition and physical activity in child care: results from an environmental intervention. Am J Prev Med. 2008;35(4):352–6.
Hardy LL, King L, Kelly B, Farrell L, Howlett S. Munch and move: evaluation of a preschool healthy eating and movement skill program. Int J Behav Nutr Phys Act. 2010;7(1):80.
Alkon A, Crowley A, Neelon S, Hill S, Pan Y, Nguyen V, et al. Nutrition and physical activity randomized control trial in child care centers improves knowledge, policies, and children's body mass index. BMC Public Health. 2014;14:215.
Metz A, Naoom, SF, Halle T, Bartley L. An integrated stage-based framework for implementation of early childhood programs and systems (OPRE Research Brief OPRE 201548). Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services. 2015. http://www.acf.hhs.gov/sites/default/files/opre/es_cceepra_stage_based_framework_brief_508.pdf. Accessed 06 Oct 2015.
Halle T, Metz A, Martinez-Beck I. Applying implementation science in early childhood programs and systems. Baltimore, USA: Brookes Publishing Co; 2013.
Rabin BA, Glasgow RE, Kerner JF, Klump MP, Brownson RC. Dissemination and implementation research on community-based cancer prevention: a systematic review. Am J Prev Med. 2010;38(4):443–56.
Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. PLoS Med. 2010;7(3):e1000251.
Jones J, Wolfenden L, Wyse R, Finch M, Yoong SL, Dodds P, et al. A randomised controlled trial of an intervention to facilitate the implementation of healthy eating and physical activity policies and practices in childcare services. BMJ Open. 2014;4(4):e005312.
Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):50.
Bero LA, Grilli R, Grimshaw JM, Harvey E, Oxman AD, Thomson MA. Closing the gap between research and practice: an overview of systematic reviews of interventions to promote the implementation of research findings. BMJ. 1998;317(7156):465–8.
Grimshaw J, Thomas R, MacLennan G, Fraser C, Ramsay CR, Vale L, et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess. 2004;8(6):1–72. iii-iv.
Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE. Knowledge translation of research findings. Implement Sci. 2012;7(1):50.
Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L, et al. Changing provider behavior: an overview of systematic reviews of interventions. Med Care. 2001;39(2):II2–II45.
Yamey G. Scaling up global health interventions: a proposed framework for success. PLoS Med. 2011;8(6), e1001049.
Rohrbach LA, Grana R, Sussman S, Valente TW. TYPE II translation transporting prevention interventions from research to real-world settings. Eval Health Prof. 2006;29(3):302–33.
Rohrbach LA, Graham JW, Hansen WB. Diffusion of a school-based substance abuse prevention program: predictors of program implementation. Prev Med. 1993;22(2):237–60.
Jones R, Riethmuller A, Hesketh K, Trezise J, Batterham M, Okely A. Promoting fundamental movement skill development and physical activity in early childhood settings: a cluster randomized controlled trial. Pediatr Exerc Sci. 2011;23:600–15.
Forsetlund L, BjØrndal A, Rashidian A, Jamtvedt G, O’Brien MA, Wolf F, et al. Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2009;2(2):CD003030.
Oxman AD, Thomson MA, Davis DA, Haynes RB. No magic bullets: a systematic review of 102 trials of interventions to improve professional practice. Can Med Assoc J. 1995;153(10):1423.
O'Brien MA, Rogers S, Jamtvedt G, Oxman AD, Odgaard-Jensen J, Kristoffersen DT, et al. Educational outreach visits: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2007;17(4):CD000409.
Soumerai SB, Avorn J. Principles of educational outreach [“academic detailing”] to improve clinical decision making. JAMA. 1990;263(4):549–56.
Alvero AM, Bucklin BR, Austin J. An objective review of the effectiveness and essential characteristics of performance feedback in organizational settings [1985–1998]. J Organ Behav Manag. 2001;21(1):3–29.
Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;6.
Noar SM, Benac CN, Harris MS. Does tailoring matter? Meta-analytic review of tailored print health behavior change interventions. Psychol Bull. 2007;133(4):673.
Finch M, Wolfenden L, Falkiner M, Edenden D, Pond N, Hardy LL, et al. Impact of a population based intervention to increase the adoption of multiple physical activity practices in centre based childcare services: a quasi experimental, effectiveness study. Int J Behav Nutr Phys Act. 2012;9(1):101–13.
Dodds P, Wyse R, Jones J, Wolfenden L, Lecathelinais C, Williams A, et al. Validity of a measure to assess healthy eating and physical activity policies and practices in Australian childcare services. BMC Public Health. 2014;14:572.
Ball S, Benjamin S, Ward D. Development and reliability of an observation method to assess food intake of young children in child care. J Am Diet Assoc. 2007;107:656–61.
Gubbels J, Kremers S, Stafleu A, Dagnelie P, De Vries N, Thijs C. Child-care environment and dietary intake of 2-and 3-year-old children. J Hum Nutr Diet. 2010;23(1):97–101.
Erinosho T, Dixon L, Young C, Brotman L, Hayman L. Nutrition practices and children's dietary intakes at 40 child-care centers in New York City. J Am Diet Assoc. 2011;111(9):1391–7.
Foodworks 7 Professional. Xyris Software Inc, Australia. 2012. http://www.xyris.com.au/. Accessed 30 Jan 2015.
National Health and Medical Research Council. Australian Dietary Guidelines. Canberra, Australia.2013. https://www.nhmrc.gov.au/guidelines-publications/n55. Accessed 30 Jan 2015.
Australian Health Survey. Users’ guide, 2011-13—discretionary food list. Canberra, Australia: Australian Bureau of Statistics; 2014. http://www.abs.gov.au/AUSSTATS/abs@.nsf/DetailsPage/4363.0.55.0012011-13?OpenDocument. Accessed on 30 Jan 2015.
McKenzie T. SOPLAY system for observing play and leisure activity in youth. Description and procedures manual. San Diego, California: San Diego State University; 2006. http://activelivingresearch.org/sites/default/files/SOPLAY_Protocols.pdf. Accessed 30 Jan 2015.
Saint-Maurice P, Welk G, Ihmels M, Richards KJ. Validation of the SOPLAY direct observation tool with an accelerometry-based physical activity monitor. J Phys Act Health. 2011;8(8):1108.
Berg S. Children’s activity levels in different playground environments: an observational study in four Canadian preschools. Early Childhood Educ J. 2014;43(4):281–7.
Janseen I, LeBlanc A. Systematic review of the health benefits of physical activity and fitness in school-aged children and youth. Int J Behav Nutr Phys Act. 2010;7:40.
Treweek S, Zwarenstein M. Making trials matter: pragmatic and explanatory trials and the problem of applicability. Trials. 2009;10:37.
Rothwell P. Factors that can affect the external validity of randomised controlled trials. PLoS Clin Trials. 2006;1(10):e9.
New South Wales Ministry of Health. Munch & move. 2015. https://www.healthykids.nsw.gov.au/campaigns-programs/about-munch-move.aspx. Accessed 30 Jan 2015.
Wiggers J, Wolfenden L, Campbell E, Gillham K, Bell C, Sutherland R, et al. Good for Kids. Good for Life. 2006–2010: evaluation report. New South Wales Ministry of Health. 2013. http://www.health.nsw.gov.au/research/Publications/good-for-kids.pdf. Accessed 30 Jan 2015.
Nathan N, Wolfenden L, Bell A, Wyse R, Morgan P, Butler M, et al. Effectiveness of a multi-strategy intervention in increasing the implementation of vegetable and fruit breaks by Australian primary schools: a non-randomized controlled trial. BMC Public Health. 2012;12(1):651.
Wolfenden L, Finch M, Nathan N, Weaver N, Wiggers J, Yoong S, et al. Factors associated with early childhood education and care service implementation of healthy eating and physical activity policies and practices in Australia: a cross-sectional study. Transl Behav Med. 2015;5(3):327–34.
The research team acknowledges the funding support of the Australian National Preventive Health Agency (reference 95WOL2011), Hunter New England Population Health and Hunter Medical Research Institute. The research team would also like to acknowledge the staff involved in delivering the intervention to participating services, in particular Paula Bridge, Tracey Findlay and Rebecca Thomson, and the staff involved in collecting the trial outcome data and participating childcare services.
The authors declare that they have no competing interests.
The first author JJ led the development of this manuscript. LW conceived the intervention concept. JJ, RW, MFi, JW, SLY, JH, AF, PD, TCM, MFr, PM, KG and LW contributed to the research design and trial methodology and advised on implementation of the intervention. JM, MFa and NP delivered the intervention. CL and JJ conducted the data analyses. All the authors contributed to and approved the final version of the manuscript.
About this article
Cite this article
Jones, J., Wyse, R., Finch, M. et al. Effectiveness of an intervention to facilitate the implementation of healthy eating and physical activity policies and practices in childcare services: a randomised controlled trial. Implementation Sci 10, 147 (2015). https://doi.org/10.1186/s13012-015-0340-z
- Obesity prevention
- Healthy eating
- Physical activity