Evaluating Early Childhood Intervention Programs: Comments on Kitzman, Knitzer, and Lipman and Boyle
Pamela Kato Klebanov, PhD
National Center for Children and Families, Teachers College, Columbia University, USA
In recent years, several reviewers have evaluated the effects of early childhood education (ECE) programs for children.1-6 Most of the research has focused on gains in children’s cognitive development rather than behavioural development.6 Overall, the results suggest that centre-based ECE programs that began in infancy have resulted in the most consistent improvements in children’s cognitive and behavioural outcomes.1,2,5
These papers examine the effects of early childhood intervention programs on the social and emotional development of young children from low-income families. Knitzer provides an overall review on interventions, Kitzman focuses on the results of home visiting programs, and Lipman and Boyle focus on barriers to the receipt of services for young Canadian children. Taken together, these papers underscore the need to focus on the behavioural and emotional outcomes for poor children, the need for more rigorous evaluation of program intensity and attention to study attrition problems, and the need for long-term follow-up of children.
Research and Conclusions
These papers highlight the diversity of the interventions conducted over the past decades.
It is impressive that reductions in behaviour problems are generally found across the different modalities. Reductions in child behaviour problems last at least six months to a year after the intervention. While most studies have focused upon and found evidence of short-term effects, I believe there is notable research involving long-term effects, such as the High/Scope Perry Preschool Project, which found effects on delinquency at age 14 and less involvement with the criminal justice system at ages 19 and 27.5,7 Retrospective Head Start data also show that those who attended Head Start were less likely to have been charged with a crime as an adult.8
Second, home visiting programs alone have less consistently demonstrated intervention effects for children. Home visiting programs tend to be more parent-focused, and thus have had more success altering parenting behaviour. However, there have been exceptions. The Nurse Home Visitation Program found effects for mothers as well as for behavioural outcomes of teenage children.9 However, home visits in combination with centre-based care have been more successful in reducing children’s behaviour problems.10,11
Third, although interventions have had an effect on parental behaviour, Knitzer, in particular, finds that there has not been a corresponding change in maternal depression. On the contrary, I believe studies have demonstrated effects for maternal depression. When the results of programs that have used random assignment to groups and provided family-oriented services to families through home visits are examined, mothers who received the treatment reported less depressive affect.9,12-16 In my opinion, what deserves examination is whether maternal characteristics such as depression mediate the association between treatment and children’s behaviour problems. To date, few studies have directly examined this question.16-19
Although there is consensus that families who experience multiple risks are the most likely to benefit from intervention programs, this population is not often served. Most of the work has been general interventions that promote child and family well-being. However, even if at-risk families are served, they are more likely to leave the program or less likely to participate. As noted by Lipman and Boyle, the availability of services in impoverished neighbourhoods, the accessibility of services and the psychological barriers to these services all present challenges to research. Finally, whether and how the intervention can be effective in the face of multiple family risks is an issue that reverberates throughout these papers and the research literature.20 The authors point out the greater success of multidimensional (compared to unidimensional) programs, however.
Implications for Development and Policy
These reviews signal the need for coherent research. The diversity of interventions conducted makes it difficult to draw any firm conclusions about what works and why. Studies need to provide more careful documentation of such information as the amount of time spent on various activities, and rely on the same curriculum approaches across sites. Existing research efforts can improve by first assessing the intensity of the program and a family’s engagement in the program. To date, few studies have examined how the amount of intervention received influences the effects of such programs. Whether there is a minimum number of visits necessary to have an effect can be analyzed in two ways: by comparing those who used the intervention to those who did not, or by measuring how the relative level of participation predicts the size of treatment effects. A few different evaluations have used these approaches to show that program effects depend on the level of program participation.21-26 Thus, programs such as home visiting can improve children’s outcomes if services are sufficiently intensive. Few studies have examined family engagement in the intervention program.27-28 However, they have found that mother and child involvement was associated with better child outcomes. Moreover, involvement by the home visitor may moderate intervention effects. Home visitors who help a mother learn more adaptive problem-solving skills and become involved in her daily life have had a positive effect on emotional health.16
If interventions are to be effective, physical and psychological barriers to services must be overcome. Even when services are available, they are not accessible. If parents are to utilize services, they need help with child care, transportation and flexibility in the hours and location of services. Some studies have dealt successfully with accessibility problems.29 Lipman and Boyle suggest that consumer research marketing techniques may be useful in identifying program preferences. Even if these barriers are overcome, psychological barriers remain. A lack of trust or confidence in service-providers or community institutions may prevent the use of services. In the case of behavioural problems, stigma presents an additional psychological barrier.
Future intervention programs need to resist financial pressures. Kitzman contends that interventions that have had a broad range of effects required significant resources and there is constant pressure to reduce the number of resources involved in implementation. However, economists such as Barnett have argued against under-investment in children, citing, for example, studies such as the Perry Preschool Project, in which benefits outweighed costs by a factor of seven to one.1,30
Intervention programs need to examine the interplay between social and emotional development and success in school. I would also add that observational measures on children’s task engagement, persistence and enthusiasm are needed to supplement the existing measures.
Finally, the authors suggest that a more modest view of program effects needs to be adopted. There is a need to re-examine the general question of what it is reasonable to expect from any given intervention.20 Many families face persistent poverty and multiple risk factors. One intervention alone cannot be expected to significantly alter their life trajectory. However, what is reasonable in terms of the size of the effects? Since cognitive effects are generally larger than behavioural effects, this expectation varies by outcome. What is reasonable in terms of duration of the effect? What is reasonable in terms of the overall breath or scope of the effect? Are effects expected for both children and parents? Are effects expected across cognitive, behavioural and health domains?
Overall, I agree with the authors that the field of early childhood intervention is still in its infancy with respect to determining the relative importance of any specific characteristic concerned. However, the fact that such programs most benefit those families facing multiple risks indicates that these programs have accomplished what they were set up to achieve.
- Barnett WS. Long-term effects of early childhood programs on cognitive and school outcomes. Future of Children 1995;5(3):25-50.
- Brooks-Gunn J, McCarton CM, Casey PH, McCormick MC, Bauer C, Bernbaum JC, Tyson J, Swanson M, Bennett FC, Scott DT, Tonascia J, Meinert CL. Early intervention in low-birth-weight premature infants: Results through age 5 years from the Infant Health and Development Program. JAMA - Journal of the American Medical Association 1994;272(16):1257-1262.
- Bryant DM, Maxwell K. The effectiveness of early intervention for disadvantaged children. In: Guralnick MJ, ed. The effectiveness of early intervention. Baltimore, Md: P.H. Brookes Publishing; 1997:23-46.
- Farran DC. Another decade of intervention for children who are low income or disabled: What do we know now? In: Shonkoff JP, Meisels SJ, eds. Handbook of early childhood intervention. 2nd ed. New York, NY: Cambridge University Press; 2000:510-548.
- Karoly LA, Greenwood PW, Everingham SS, Hoube J, Kilburn MR, Rydell CP, Sanders M, Chiesa J. Investing in our children: What we know and don’t know about the costs and benefits of early childhood interventions. Santa Monica, Calif: RAND; 1998. Available at: http://www.rand.org/publications/MR/MR898/. Accessed November 25, 2004.
- Yoshikawa H. Long-term effects of early childhood programs on social outcomes and delinquency. Future of Children 1995;5(3):51-75.
- Schweinhart LJ, Barnes HV, Weikart DP, Barnett WS, Epstein AS. Significant benefits: The High/Scope Perry Preschool Study through age 27. Ypsilanti, Mich: High/Scope Press; 1993.
- Garces E, Thomas D, Currie J. Longer term effects of Head Start. American Economic Review 2002;92(4):999-1012.
- Olds DL, Henderson CR, Kitzman HJ, Eckenrode JJ, Cole RE, Tatelbaum RC. Prenatal and infancy home visitation by nurses: Recent findings. Future of Children 1999;9(1):44-65.
- Gross RT. Enhancing the outcomes of low-birth-weight, premature infants: A multisite, randomized trial. JAMA - Journal of the American Medical Association 1990;263(22):3035-3042.
- Kisker EE, Paulsell D, Love JM, Raikes H. Pathways to quality and full implementation in Early Head Start programs. Washington, DC: U.S. Department of Health and Human Services; 2002. Available at: http://www.acf.hhs.gov/programs/opre/ehs/ehs_resrch/reports/pathways/pathways.pdf. Accessed January 21, 2005.
- Barnard KE, Magyary D, Sumner G, Booth CL, Mitchell SK, Spieker SJ. Prevention of parenting alterations for women with low social support. Psychiatry 1988;51(3):248-253.
- Booth CL, Mitchell SK, Barnard KE, Spieker SJ. Development of maternal social skills in multiproblem families: Effects on the mother-child relationship. Developmental Psychology 1989;25(3):403-412.
- Erickson MF, Korfmacher J, Egeland BR. Attachments past and present: Implications for therapeutic intervention with mother infant dyads. Development and Psychopathology 1992;4(4):495-507.
- Olds DL, Henderson CR, Kitzman H. Does prenatal and infancy nurse home visitation have enduring effects on qualities of parental caregiving and child health at 25 to 50 months of life? Pediatrics 1994;93(1):89-98.
- Klebanov PK, Brooks-Gunn J, McCormick MC. Maternal coping strategies and emotional distress: Results of an early intervention program for low birth weight young children. Developmental Psychology 2001;37(5):654-667.
- Burchinal MR, Campbell FA, Bryant DM, Wasik BH, Ramey CT. Early intervention and mediating processes in cognitive performance of children of low-income African American families. Child Development 1997;68(5):935-954.
- NICHD Early Child Care Research Network. Child care and mother-child interaction in the first 3 years of life. Developmental Psychology 1999;35(6):1399-1413.
- Harnish, JD, Dodge KA, Valente E, Bierman KL, Coie JD, Dodge KA, Greenberg MT, Lochman JE, McMahon RJ. Mother-child interaction quality as a partial mediator of the roles of maternal depressive symptomatology and socioeconomic status in the development of child behavior problems. Child Development 1995;66(3):739-753.
- Brooks-Gunn J. Do you believe in magic?: What we can expect from early childhood interventions programs. Social Policy Report 2003;17(1):3-14. Available at: http://www.srcd.org/Documents/Publications/SPR/spr17-1.pdf. Accessed November 25, 2004.
- Brooks-Gunn J, Burchinal M, Lopez M. Enhancing the cognitive and social development of young children via parent education in the Comprehensive Child Development Program. National Center for Children and Families, Teachers College, Columbia University; 2002. Unpublished manuscript.
- Hill J, Waldfogel J, Brooks-Gunn J. Differential effects of high-quality child care. Journal of Policy Analysis and Management 2002;21(4):601-627.
- Osofsky JD, Culp AM, Ware LM. Intervention challenges with adolescent mothers and their infants. Psychiatry 1988;51(3):236-241.
- Kitzman H, Olds DL, Henderson CR, Hanks C, Cole R, Tatelbaum R, McConnochie KM, Sidora K, Luckey DW, Shaver D, Engelhardt K, James D, Barnard K. Effect of prenatal and infancy home visitation by nurses on pregnancy outcomes, childhood injuries, and repeated childbearing trial: A randomized controlled trial. JAMA - Journal of the American Medical Association 1997;278(8):644-652.
- Ramey CT, Bryant DM, Wasik BH, Sparling JJ, Fendt KH, LaVange LM. Infant Health and Development Program for low birth weight, premature infants: Program elements, family participation, and child intelligence. Pediatrics 1992;89(3):454-465.
- Sparling JJ, Lewis I, Ramey CT, Wasik BH, Bryant DM, LaVange LM. Partners: A curriculum to help premature, low birthweight infants get off to a good start. Topics in Early Childhood Special Education 1991;11(1):36-55.
- Berlin LJ, O’Neal CR, Brooks-Gunn J. What makes early intervention programs work? The program, its participants, and their interaction. Zero to Three 1998;18(4):4-15.
- Liaw FR, Meisels SJ, Brooks-Gunn J. The effects of experience of early intervention on low birth weight, premature children: The Infant Health & Development Program. Early Childhood Research Quarterly 1995;10(4):405-431.
- Constantine WL, Haynes CW, Spiker D, Kendall-Tackett K, Constantine NA. Recruitment and retention in a clinical trial for low birth weight, premature infants. Journal of Developmental and Behavioral Pediatrics 1993;14(1):1-7.
- Gomby DS, Larner MB, Stevenson CS, Lewit EM, Behrman RE. Long-term outcomes of early childhood programs: Analysis and recommendations. Future of Children 1995;5(3):6-24.
How to cite this article:
Klebanov PK. Evaluating Early Childhood Intervention Programs: Comments on Kitzman, Knitzer, and Lipman and Boyle. In: Tremblay RE, Boivin M, Peters RDeV, eds. Encyclopedia on Early Childhood Development [online]. http://www.child-encyclopedia.com/low-income-and-pregnancy/according-experts/evaluating-early-childhood-intervention-programs-comments. Updated August 2007. Accessed July 22, 2019.