Araujo Etal2021
Araujo Etal2021
Araujo Etal2021
ABSTRACT
Most evidence on the effects of policies to promote child development refers to pilot programs implemented under tightly-controlled circumstances. We provide
novel evidence on the effects of home visiting delivered at scale. The program we study, Cuna Mas in Peru, was started from scratch in 2012 and, within 3
years, was delivering weekly home visits to over 67,000 children in rural areas. Identification comes from random assignment of municipalities to treatment and
control conditions. We show that, after approximately 2 years, children randomly assigned to treatment have child development scores that are 0.10 standard
deviations higher than those assigned to control. The estimated benefit–cost ratio of the intervention is 5.4.
✩ The authors would like to thank the Government of Peru, in particular the Ministries of Economics and Finance and of Social Inclusion and Development
and the Cuna Mas Program, for valuable discussions and for the support to this evaluation. We would also like to thank all the families, Cuna Mas staff, and field
teams involved in the study. Finally, special thanks to Erika Dunkelberg, María Adelaida Martínez, Rafael Novella, Beatriz Ore, Julieth Parra and Romina Tomé
for outstanding technical and research assistance at various stages of the project. The views here presented do not represent the Inter-American Development
Bank, The World Bank, their boards of directors, or the countries they represent.
∗ Corresponding author.
E-mail address: [email protected] (N. Schady).
1
Reviews of the literature by economists include Almond and Currie (2011), Almond et al. (2018), Attanasio (2015) and Heckman and Mosso (2014).
2
There are 1854 districts in Peru. As we discuss below, Cuna Mas home visiting services were not meant to reach all districts.
3
In the U.S., Early Head Start (EHS) reaches approximately 250,000 children through hundreds of grantees or individual providers (https://eclkc.ohs.acf.hhs.
gov/programs/article/about-early-head-start-program, consulted on April 1, 2018). EHS has different modalities. In 2013, 42 percent of children were in the
home-based modality, in which services are delivered in the family’s home from a home visitor and through group activities (Mayoral, 2014). In developing
countries, other large home visiting programs include the Creciendo con Nuestros Hijos program in Ecuador, which reaches approximately 200,000 children
(direct communication with Ministry of Social and Economic Inclusion, 2018), and the Criança Feliz program in Brazil, which reaches 233,000 families (direct
communication with Ministry of Social Development, 2018).
https://doi.org/10.1016/j.pubecp.2021.100003
Received 25 October 2020; Received in revised form 7 April 2021; Accepted 19 April 2021
2666-5514/© 2021 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license
(http://creativecommons.org/licenses/by-nc-nd/4.0/).
M.C. Araujo, M. Dormal, S. Grantham-McGregor et al. Journal of Public Economics Plus 2 (2021) 100003
home visits, are 0.15 standard deviations. A back-of-the-envelope cal- that Cuna Mas faced, and speculate on how these may have affected
culation suggests that the benefit–cost ratio of the intervention, 5.4, program impacts.
was positive and sizeable. The rest of the paper proceeds as follows. In Section 2 we discuss
Consistent with the improvements in child development, we find the Cuna Mas home visiting program, including how it built on earlier
that Cuna Mas changed parenting behaviors: Parents randomly assigned programs. Section 3 discusses the evaluation design and data. We
to the early treatment group played more often with their children, present results in Section 4, and conclude in Section 5.
and were less likely to engage in punitive parenting strategies (like
hitting or yelling at a child). In addition, among children 36 months 2. The Cuna Mas home visiting program
and older (the age at which they are no longer eligible for home visits
but become eligible for publicly-funded preschool), children randomly Home visiting programs that seek to improve child development
assigned to the Cuna Mas treatment were 8 percentage points more have been of interest to policymakers and researchers in developing
likely to be enrolled in preschool, relative to preschool enrollment rates countries for over two decades. There are a handful of randomized eval-
of 50 percent in the control group. uations of these programs, mainly published in the medical literature
This paper makes several contributions. First, our paper contributes (see Aboud and Yousafzai, 2015 for a review). All these evaluations
to a small literature on the impact of home visiting programs. Relative refer to efficacy trials or small-scale pilots, as opposed to at-scale
to other interventions for young children – for example, income trans- interventions.6
fers, or access to preschool – less is known about the effectiveness of The best-known evaluation of a home visiting program in a develop-
programs that directly seek to improve parenting practices. And yet, ing country began in 1986 in Kingston, Jamaica. The program focused
home visiting programs are often the only way of reaching children in on improving child outcomes through psychosocial stimulation and
remote, rural areas of developing countries—precisely those children play. It was delivered by paraprofessional community health workers.
who are most vulnerable and tend to have the largest developmental Children in the treatment group received weekly home visits beginning
delays. at 9–24 months of age, and for 2 years thereafter.
Second, our paper adds to a literature on the best use of local The Jamaican intervention has been evaluated in a number of
resources to effect behavioral change. Cuna Mas home visitors were randomized control trials. The most ambitious is a trial of 129 children
chosen from the same villages as the families they would be working who, to date, have been followed for over 20 years. Early results found
with. This was done to build trust between families and visitors, and very large, short-run (after 24 months) effects on child cognition, about
to reduce program costs. However, in the areas where Cuna Mas 0.88 standard deviations (Grantham-McGregor et al., 1991; Walker
operated, average education levels were often quite low—and so, as a et al., 2000). Approximately 20 years after the program ended, those
result, were those of the home visitors: 56 percent had not completed who had been randomly assigned to treatment had completed more
secondary school. We show that, despite the low education levels of schooling, had higher test scores, were more likely to be employed,
home visitors, the program had a positive, albeit modest, effect on child and had earnings that were 25 percent higher than those assigned to
development.4 the control group; they also had improved cognitive, personal-social
Third, and perhaps most importantly, our paper speaks to the and mental health outcomes (Gertler et al., 2014; Walker et al., 2011).
challenges of scaling up promising interventions.5 As we discuss below, The Jamaican model – now referred to as Reach Up (Walker et al.,
the intervention we study built on a very small efficacy trial in Jamaica 2018) – has been adapted and piloted in a number of countries. One
(with 64 children receiving home visits), and a pilot carried out in such adaptation took place in Colombia. There, researchers imple-
Colombia (with 720 children receiving visits). By contrast, within mented a program with a curriculum that was very similar to that used
3 years, the Cuna Mas program was delivering home visits to almost in Jamaica. However, home visits in Colombia were delivered by com-
100 times as many children as those in the Colombian pilot. Much munity mothers (rather than health workers). These mothers, known as
can go wrong in the transition from a small pilot, covering a few madres líderes (leader mothers), had been elected by their communities
dozen or a few hundred children, to an at-scale intervention that covers to be liaisons with the local administrators of a nationwide cash transfer
tens of thousands. We discuss some of the implementation difficulties program. They were chosen to be home visitors because, in the words
of the study team, they were ‘‘influential and well-connected in their
4 communities’’. Working with community personnel was done to make
Structured (as opposed to informal) peer-to-peer mentoring and coaching
have been widely used in education and health—for example, programs in the intervention scalable and potentially sustainable beyond the life of
which ‘‘expert’’ teachers or nurses observe and provide feedback to other the project. About 720 families received home visits for 18 months.
(often rookie) teachers or nurses. Mentoring and coaching is also an important A randomized evaluation of the Colombian pilot found positive effects
element in various programs for youth. For example, in the Becoming A on child development in the short run, about 0.18 standard deviations
Man program, which sought to reduce crime and school dropout among (Attanasio et al., 2014). However, no evidence of significant effects was
disadvantaged youth in Chicago, counselors working with youth were fre- detected 2 years after the program had ended (Andrew et al., 2018).
quently hired from neighborhoods very similar to those in which they would Cuna Mas built directly on the Jamaican (and especially) Colombian
be working (Heller et al., 2017). However, in these programs the mentors experiences.7 It focused on families with children 0–36 months old,
tend to be highly skilled. There are also many examples of programs that and it was targeted geographically to districts with high levels of
use local facilitators or promoters to change behaviors, including programs
poverty and stunting, and a population that was mainly rural.8 These
to promote clean water use (Zwane and Kremer, 2007), breastfeeding (see
were the same districts where a different welfare program (Juntos,
Chapman et al., 2010; Haroon et al., 2013; Rollins et al., 2016), adoption
of insecticide treated nets and other strategies to prevent malaria (see Salam
et al., 2014 for a review), or health-seeking behaviors for parents of young 6
There is also a literature on home visiting programs in the U.S. Olds
children (see Prost et al., 2013 for a review and meta-analysis), but these (2010) is a review of evaluations of the Family-Nurse Partnership (NFP). Love
programs generally do not use a structured curriculum or rely on regular et al. (2013) reports results from a randomized evaluation of EHS. Some EHS
home visits. See also Björkman Nyqvist et al. (2019) for a recent example sites include home-based services.
7
in the economics literature. To change parenting behaviors in a scalable and See Araujo et al. (2021) for a discussion of the differences in design and
sustainable way, Cuna Mas designed an intervention that, on the one hand, implementation between the Jamaican efficacy trial, the Colombian pilot, and
was more structured than most programs that rely on local promoters but, on the Cuna Mas at-scale intervention.
8
the other hand, was built around home visitors who were less skilled than is To be eligible, districts had to have a poverty rate of 50 percent or higher,
the case in most coaching and peer-to-peer mentoring programs. a stunting rate among children 0–5 years old of 30 percent or higher, and
5
On the challenges to scale-up, see, especially, Al-Ubaydli et al. (2017), more than 50 percent of the population living in rural villages. (Peru defines
and the collection of papers in List et al. (2021); also, Banerjee et al. (2017). rural villages as those with a population of at most 2,000, or less than 400
2
M.C. Araujo, M. Dormal, S. Grantham-McGregor et al. Journal of Public Economics Plus 2 (2021) 100003
a cash transfer program) also operated. The two programs were run low-cost home-made toys, blocks, picture books, puzzles, and sorting
separately, and there was no attempt to coordinate them in any explicit and matching games, among others. Activities were organized in ac-
way (beyond the fact that they were targeted at the same districts). tivity cards (‘fichas’), arranged in order of difficulty, with a suggestion
Juntos was created long before Cuna Mas, and the proportion of Juntos of a child age range (1–2 months) when they would be appropriate.
beneficiaries in treatment and control groups was very similar.9 This facilitated scaffolding—that is, the practice of teaching a child
Cuna Mas was designed to deliver weekly, hour-long, play-based something new by building on something she already knows. Activities
home visits by paraprofessionals with limited education. As in Colom- and games generally supported the development of cognitive abilities
bia, home visitors were drawn from the same communities as the (e.g., object permanence, causation, problem solving) and language
families they would work with. They were primarily women, and each (imitation of gestures and vocalization, introduction of new words).
worked with 10 families. Home visitors received a modest stipend For older children, there were additional activities to teach concepts
for their work and, for most of them, this was their only paid job. (size, quantity, color, shape, and position, amongst others), and early
Although they were in principle required to have completed secondary executive functions (such as attention and persistence). All materials
school, in practice only 44 percent were secondary school graduates. were adapted to the local context, and all images in the sorting and
Supervisors, in turn, were required to have tertiary education. Each matching games, picture books, and puzzles were drawn to reflect the
supervisor worked with 10 home visitors. Twice a month, supervisors diversity of both the Andean and Amazonian cultures of Peru.11
were meant to work individually with each of the home visitors they Cuna Mas home visits were based on demonstration and practice.
supervised, accompanying them on their visits, observing them, provid- During the visit, the home visitor modeled interactions with the child,
ing feedback, and helping with the planning of activities for individual encouraged and coached the caregiver to engage in play and con-
children. versation with her child and respond to vocalizations and actions by
Although Cuna Mas was explicitly modeled on the Colombian pilot, the child. The visitor demonstrated praising the child, and celebrating
there were important differences between the two programs. Cuna Mas her efforts and achievements as a way of promoting child self-esteem
was truly an intervention at scale—with all the attendant implementa- and socio-emotional development. Verbal or corporal punishment to
tion challenges. There were no madres líderes in Peru, so Cuna Mas had redirect child behaviors was discouraged. Special emphasis throughout
to develop its own selection criteria for home visitors.10 Some aspects the visit was placed on listening to the caregiver, seeking her opinion,
of the intervention were watered down: Pre-service training was much
giving her encouragement, and praising both caregiver and child.
more limited in Peru both for home visitors – 4 days in Peru, 3 weeks in
A typical Cuna Mas visit began by catching up with the family and
Colombia – and for supervisors — 9 days in Peru, 6 weeks in Colombia.
washing hands. The home visitor then engaged the mother and child
Home visitors in both countries used a structured curriculum of
in a play session structured in three separate sections (‘moments’):
developmentally-appropriate activities, rich in play materials—mostly
(i) Family Life (Vida en familia), which covered a play-based activity
that could be embedded into daily routines—for example, ‘words
dwellings.) Within these districts, Cuna Mas operates only in rural villages.
I can learn during bath time’ or ‘sorting clothes by type and size
Cuna Mas also provides childcare in low-income urban areas—see Araujo et al.
(2019). However, there is no overlap in the coverage of these two services: while doing laundry’;
Cuna Mas did not provide childcare in any of the districts that are the basis (ii) I Learn while Playing (Jugando aprendo), which introduced a
for the analysis in this paper. game or activity oriented towards achieving a specific cognitive
9
At baseline, 52 percent of households in the treatment group, and 47 or motor task, often using a designated toy or material (for
percent in the control group, received transfers. If cash transfers and home example, a puzzle); and
visits are complements, part of the Cuna Mas effect we estimate could in (iii) Tell Me a Story (Cuéntame un cuento), which covered a language
principle be a result of the interaction between Juntos and Cuna Mas. This is activity, most often using a picture book or related material with
because, strictly speaking, we evaluate the effect of home visits in a context in
images to talk about.
which households were already eligible for a cash transfer program. However,
we provide various suggestive pieces of evidence that this is unlikely to The methodology and structure of each ‘moment’ was the same. First,
be the main reason for the positive effects of Cuna Mas. First, randomized the home visitor asked how caregiver and child had progressed with
evaluations of programs like Juntos indicate that, on their own, cash transfers
the activities from the previous visit and, ideally, caregiver and child
have positive, but only modest effects on child development (see Paxson and
showed their progress. Then, the home visitor introduced the new
Schady, 2010 on Ecuador; and Macours et al., 2012 on Nicaragua). Second, we
run regressions of child development on random assignment to the Cuna Mas
activity for the week, describing it, demonstrating it, and watching
treatment, an indicator variable for whether a household received Juntos at child and caregiver replicate and adapt it. The visit ended with a song,
baseline, and the interaction between the two, as well as trio fixed effects. The a review of activities to continue during the week, and encouragement
interaction effect is −0.040 (with a standard error of 0.063). Finally, we check to incorporate them in daily routines.
whether being randomly assigned to the Cuna Mas treatment group increased
Juntos take-up. We find no evidence that this is the case. In a multinomial 3. The Cuna Mas evaluation
logit regression with four categories (receiving cash transfers at baseline and
follow-up, not receiving cash transfers in either time period, receiving transfers
3.1. Evaluation design
at baseline but not at follow-up, and receiving transfers at follow-up but not at
baseline), the marginal effects of the Cuna Mas treatment on all four categories
are small and insignificant. In the case of switching from not receiving to When the Peruvian government designed Cuna Mas, it understood
receiving Juntos transfers, the marginal effect associated with the Cuna Mas that, because of operational and budget constraints, it could not reach
treatment is −0.028 (with a standard error of 0.018). all the population eligible for the program immediately. We designed
10
Home visitors were selected with input from the community. Specifically, and implemented an experimental evaluation that took advantage of
community leaders proposed candidates who met the following criteria: they the gradual roll-out of the intervention.
were 21 or older, literate, had past experience working with children, were Specifically, districts eligible for Cuna Mas were ordered by their
fluent in local languages, and were respected by others in the community. Cuna
poverty rate.12 Sixty districts were randomly selected, one at a time,
Mas supervisors reviewed the proposed candidates and evaluated them, in
some cases with community leaders. There was a manual and a checklist they
were meant to use in the process (including a mental health screening, and 11
In order to increase appropriation of the intervention by families and
an assessment of communication and inter-personal skills). We do not know visitors, two independent sets of materials were produced, one per region.
12
to what extent these procedures were consistently applied in the selection of The poverty rate was calculated by the Peruvian National Statistics
home visitors. Agency, INEI. There were some additional restrictions when constructing
3
M.C. Araujo, M. Dormal, S. Grantham-McGregor et al. Journal of Public Economics Plus 2 (2021) 100003
and matched with the districts that had the next-highest and next- of receptive language. Maternal vocabulary has been shown to predict
lowest poverty level. In this way, 60 ‘‘trios’’ of districts were con- child development, even after controlling for years of completed school-
structed. Two districts were then randomly assigned to the first wave ing. This pattern has been documented in many studies, including in
of Cuna Mas home visits – our treatment group – and one to the last Ecuador, a country that neighbors Peru (Schady, 2011).
wave—our control group. Cuna Mas did not initiate operations in the
control districts before the follow-up survey of the evaluation.13 3.2.3. Children
In each of the 180 districts of the evaluation sample, we selected Child data included gender and age, height, weight, and whether a
the two rural villages with the largest number of children between 0 child lived with both parents. To measure child development, we used
and 24 months of age (according to the latest population census). We the Ages and Stages Questionnaire, Third Edition (ASQ-3; Squires et al.,
then carried out a census to identify all children in this age range in 2009). The ASQ-3 measures development in five domains—problem-
the 360 villages in the sample. From that roster, 17 children in each solving (or cognition), communication (or language), fine motor, gross
village were randomly chosen. In villages where there were fewer than motor, and personal-social. The test has been applied widely in Latin
17 children, all eligible children were included.14 America, including in earlier work on the quality of Cuna Mas daycare
services (Araujo et al., 2019). Further details on the ASQ-3 and its
3.2. Data on households, mothers, and children administration are given in Appendix A.
At baseline and endline, respondents were asked about the fre-
Baseline data were collected between April and July 2013; program quency with which they or someone else in the household engaged
implementation in the 120 treatment districts began in August 2013, in various parenting behaviors with the focal child. These questions
with some variation in the exact date on which a given village first were drawn from items in selected subscales of UNICEF’s Family Care
received home visits; a follow-up survey was conducted between May Indicators (FCI; Kariger et al., 2012) and the Home Observation of the
and December 2015. The household questionnaires at baseline and Home Environment (HOME; Caldwell and Bradley, 2001). We group
follow-up collected information on household characteristics, parenting behaviors in four categories: (i) Play activities, including reading or
behaviors, and child development. The respondent was generally the playing with children; (ii) harsh corporal punishment, including hitting,
mother of the child. spanking, or pinching the child; (iii) harsh verbal punishment, includ-
ing yelling at or scolding the child; (iv) positive responses to children,
3.2.1. Households including hugging children or praising them. We also collected data on
We collected data on basic household characteristics, including the the number of play materials available for children. Finally, at follow-
main material of the floor; the number of bedrooms; the number of up, we asked parents of children 36 months or older whether their child
assets in the household;15 the number of household members; whether was enrolled in preschool.
the household had piped water, and whether it was connected to the
sewerage system (two separate variables). 3.3. Baseline characteristics
3.2.2. Mothers Table 1 presents descriptive statistics of the 4,685 children in the
Data on mothers included their age and years of completed school- sample and their families.16 At baseline, the average child was just
ing. At baseline, we collected an adapted version of the Center for over a year old, and half of them were girls. Eighty-six percent of
Epidemiologic Studies Short Depression Scale (CESD-R-10; Radloff, the children in the sample lived with both parents. We use World
1997; Björgvinsson et al., 2013). Maternal depression has been found Health Organization (WHO) tables to calculate the proportion stunted
to be negatively associated with child development in many settings (height-for-age < 2 standard deviations in the reference population),
(Sohr-Preston and Scaramella, 2006), including in developing countries wasted (weight-for-height < 2 standard deviations), and overweight
(Walker et al., 2007). Also at baseline, mothers were administered the (weight-for-height > 2 standard deviations). Table 1 shows that a
Test de Vocabulario en Imágenes Peabody (TVIP; Dunn et al., 1986), the substantial proportion of children, 37 percent, were stunted. The pro-
Spanish version of the Peabody Picture Vocabulary Test (PPVT), a test portion wasted, on the other hand, was very low (1 percent of the
sample), as was the proportion overweight (4 percent of the sample).
The ASQ-3 manual provides cutoff scores, based on a reference
the sample. Specifically, the sample frame excluded districts in one of the (norming) population in the U.S., to categorize children into one of
following groups: (a) all districts in regions that were part of a pilot im- three groups: children with ‘‘typical development’’, those in a ‘‘moni-
plementation of Cuna Mas carried out during 2012; (b) districts where the
toring zone’’ and, those who exhibit ‘‘possible delays in development’’.
predecessor program of Cuna Mas (called Wawa Wasi) had implemented rural
Table 1 indicates that, using these ‘‘external’’ cutoffs, 26 percent of
group parenting interventions; (c) regions with so few eligible districts that
implementation was uncertain; and (d) districts that – based on the most recent children had a possible delay in at least one domain of the ASQ-3, 18
census data – did not have at least two rural villages, each with a minimum percent in at least 3 domains, and 3 percent had possible delays in all
of 21 children 0–24 months of age. 5 domains.
13
Originally, Cuna Mas sought to evaluate two treatments: one that offered Most children, 91 percent, had an adult in the household carry
weekly home visits, and a second one that supplemented them with bi-weekly out a play activity with them in the last week. This is perhaps not
group meetings. In practice, implementation of the second treatment did not surprising given that the list of play activities is long—it includes read-
occur because it demanded additional organization and resources that were ing, telling stories, singing, playing, drawing, or going for a walk with
not in place. For that reason, we merge the two treatment groups into one for the child. Fifteen percent of children were subjected to harsh corporal
our analysis.
14
punishment—a substantial proportion considering that all children in
DGSE-MIDIS et al. (2015) presents more details on the evaluation design,
sampling strategy and the power calculations. Rubio-Codina et al. (2016b)
16
compare the characteristics of the districts in this sample to the national We exclude 582 children (11 percent of the total number of children with
average. data at baseline and follow-up) from the estimation sample because they lived
15
We collected information on whether a household had the following in households that were not monolingual Spanish speakers. Children in these
assets: color TV, cable, DVD player, mobile phone, stereo equipment, com- households would be learning an indigenous language (Quechua, Aymara, or
puter, refrigerator, stove, washing machine, iron, and blender. As a result, the one of the many languages spoken in the eastern Amazon rainforest), and
number of assets in the household can range from 0 to 11. In our data, 20.1 in many cases this would be their first (or only) language, as well as their
percent of children live in households with no assets, and 0.6 percent live in mothers’. The ASQ-3 and TVIP may work poorly with children and adults in
households that report having all 11 assets. households where a language other than Spanish is spoken.
4
M.C. Araujo, M. Dormal, S. Grantham-McGregor et al. Journal of Public Economics Plus 2 (2021) 100003
Table 1
Descriptive statistics of children and their families.
N Mean/Proportion Standard deviation
Children
Age in months 4685 12.8 6.6
Proportion female 4685 0.49
Proportion with no delays in ASQ-3 4685 0.18
Proportion with delays in at least 1 ASQ-3 domain 4685 0.26
Proportion with delays in at least 2 ASQ-3 domains 4685 0.25
Proportion with delays in at least 3 ASQ-3 domains 4685 0.18
Proportion with delays in at least 4 ASQ-3 domains 4685 0.10
Proportion with delays in all 5 ASQ-3 domains 4685 0.03
Height-for-age z-score 4559 −1.66 1.07
Proportion stunted (height-for-age < -2 SD) 4559 0.37
Weight-for-height z-score 4559 0.35 0.92
Proportion wasted (weight-for-height < -2 SD) 4561 0.01
Proportion overweight (weight-for-height > 2 SD) 4561 0.04
Proportion who carried out 1 or more play activities in last week 4685 0.91
Proportion subjected to harsh corporal punishment in last week 4685 0.15
Proportion subjected to harsh verbal punishment in last week 4685 0.31
Proportion praised for good behavior in last week 4685 0.95
Proportion living with both parents 4685 0.86
Mothers
Age 4685 28.5 7.6
Years of schooling 4453 6.9 4.1
TVIP score 4363 65.8 15.3
Proportion with CESD-R score > =10 4672 0.13
Households
Number of members 4685 5.5 2.1
Number of bedrooms in home 4685 2.5 1.5
Number of assets (0-11) 4685 2.6 2.3
Proportion with piped water in dwelling 4685 0.60
Proportion connected to sewerage system in dwelling 4685 0.26
Main material of floors is earth 4685 0.73 0.44
Note: Possible delays in development were calculated according to the cutoffs provided in the ASQ-3 manual, and the nutrition
indicators following the World Health Organization guidelines. Maternal depression was measured with an adapted version
of the CESD-R-10 (Radloff, 1997; Björgvinsson et al., 2013). TVIP refers to Test de Vocabulario en Imágenes Peabody (Dunn
et al., 1986), the Spanish version of the Peabody Picture Vocabulary Test (PPVT). Assets include: color TV, cable, DVD player,
mobile phone, stereo equipment, computer, refrigerator, stove, washing machine, iron, and blender.
the sample were 2 years of age or younger at baseline. Thirty-one corresponds to an aggregate that equally weights all five domains.
percent of children were shouted at or scolded. The proportion of Results refer to children in the control group at follow-up. In each
children who were praised for good behavior at least once in the last panel, we show bars corresponding to the development of children of
week was very high, 95 percent. mothers with different education levels: incomplete primary or less (30
On average, mothers were 28 years old at baseline, and had com- percent of mothers in the control group); completed primary school
pleted 7 years of schooling. Mothers also had very low vocabulary (27 percent); incomplete secondary school (19 percent); and completed
levels. Test developers for the TVIP provide conversion tables to calcu- secondary school or more (24 percent).
late normed vocabulary scores. In the reference population for the test Fig. 1 shows clear maternal education gradients in child develop-
(in Mexico and Puerto Rico), the mean score is 100 and the standard ment. The steepest gradients are found in cognition—the difference
deviation is 15. The mean TVIP score of mothers in our sample was 66, between children of mothers with the most and least schooling is 0.56
more than two standard deviations below the mean of the reference standard deviations. This is also the domain in which, at follow-up,
population, and also below the scores observed in a population of low- the largest proportion of children in the control group, 41 percent,
income mothers in rural areas in Ecuador (mean of 73; Schady, 2011). had ‘‘possible delays in development’’. Gradients in other domains are
Thirteen percent of mothers had CESD-R-10 scores above or equal to less pronounced, especially in personal-social development.18 We note
10, the standard cutoff for depression. that, by design, families in the evaluation sample were very poor—
presumably, socioeconomic gradients in a nationally representative
Fig. 1 focuses on maternal education gradients in child develop-
sample of children would be larger.
ment. For this purpose, and for the rest of the analysis, we first convert
Further details on the evaluation sample are in Appendix A. In
the raw ASQ-3 scores for each domain to z-scores after nonparamet-
this appendix, we show that girls had higher ASQ-3 scores than boys,
rically adjusting for child age.17 The first five panels in the figure
about 0.20 standard deviations, a result that is consistent with others
correspond to individual domains in the ASQ-3, and the last panel
reported in the literature (Bando et al., 2016). Children with poor
nutritional status, in particular children with low height-for-age, had
17
In principle, child development scores can be age-standardized by cal- lower development. Maternal TVIP and household wealth (as measured
culating the mean and standard deviation of the raw score for children in by the number of household assets) are positively correlated with ASQ-
a narrow age range (say, every month of age), subtracting the mean from a 3 scores. On the other hand, maternal depression does not predict child
child’s raw score, and dividing by the standard deviation. In practice, however, development in our sample.
this procedure can be sensitive to outliers if the number of children in each
month of age is small. Rubio-Codina et al. (2016a) propose a three-step method
that is less sensitive to outliers and, as in earlier work on Cuna Mas (Araujo and take the square root of the fitted values ĝ i ; (3) finally, to calculate the age-
et al., 2019), we follow this closely: (1) First, to obtain an age-specific mean adjusted z-score for each domain, ZYi , we take each child’s raw score, subtract
score, f̂i , for each domain in the ASQ-3, we run nonparametric regressions of the mean obtained√in (1), and divide by the standard deviation obtained in
a child’s raw score, Yi , on age in days, Xi : Yi =f(Xi ) + 𝜀i ; (2) next, to obtain an (2): ZYi =(Yi -f̂ i )/ 𝑔̂i
18
age-specific standard deviation of the score, we run nonparametric regressions In every panel in Fig. 1, the difference between children of mothers with
of the square of the residuals, 𝜀i , on age of the child in days, 𝜀2i = g(Xi ) + 𝑣i , the highest and lowest levels of education is significant at the 95 percent level.
5
M.C. Araujo, M. Dormal, S. Grantham-McGregor et al. Journal of Public Economics Plus 2 (2021) 100003
Appendix A also shows that the baseline characteristics of treated of visits received was 64, and families at the 10th, 25th, 75th and
and control children are balanced. Attrition between baseline and 90th percentiles of the distribution received 27, 44, 85, and 99 visits,
follow-up was low, and is similar in treatment (8.6 percent attrited) respectively. Panel B focuses on the ratio of visits received to the
and control groups (9.3 percent attrited). The difference in attrition number of visits a family was meant to receive—what we call effective
rates between treated and control children is not significant (p-value dosage.19 The figure, which is once again limited to families that
of 0.46), and the observable characteristics of attritors in both groups received at least one visit, shows that, for just under half of families,
are statistically indistinguishable. effective dosage was between 70 and 100 percent. However, many
families received very few visits—one in ten received 30 percent or
3.4. Take-up and dosage less of the visits they were meant to receive.
6
M.C. Araujo, M. Dormal, S. Grantham-McGregor et al. Journal of Public Economics Plus 2 (2021) 100003
Table 2
Take-up and dosage.
At least one visit Number of visits Effective visits (%)
(1) (2) (3) (4) (5) (6) (7) (8) (9)
Child characteristics
Age > 36 months −0.079*** −0.085*** −0.095*** −17.801*** −18.566*** −19.135*** 2.636** 1.262 1.375
(0.022) (0.020) (0.020) (1.843) (1.752) (1.800) (1.198) (1.099) (1.161)
Gender 0.007 0.006 0.009 −1.114 −1.384 −0.973 −0.257 −0.586 −0.441
(0.019) (0.016) (0.017) (1.407) (1.195) (1.197) (0.990) (0.926) (0.984)
Baseline ASQ-3 above median −0.058*** −0.041*** −0.033* −2.392 −2.256* −1.151 1.066 0.357 0.808
(0.021) (0.015) (0.017) (1.563) (1.176) (1.251) (1.157) (0.901) (0.958)
Stunted 0.055** 0.042** 0.052** 1.043 0.336 2.442* 0.676 0.455 −0.280
(0.024) (0.019) (0.020) (1.782) (1.364) (1.377) (1.146) (0.942) (0.995)
Family characteristics
High school complete or above −0.120*** −0.094*** −0.057*** −7.391*** −6.830*** −4.017** −0.320 −1.433 −1.109
(0.031) (0.024) (0.020) (2.201) (1.794) (1.659) (1.250) (1.173) (1.300)
Mother TVIP score above median −0.051* −0.028 −0.009 −4.817** −3.354** −1.685 −1.094 −0.423 0.032
(0.027) (0.022) (0.021) (1.954) (1.674) (1.570) (1.239) (1.136) (1.238)
Mother age above median 0.023 0.004 −0.015 1.517 0.541 −0.175 1.613 2.009** 1.943*
(0.019) (0.017) (0.018) (1.454) (1.292) (1.406) (1.223) (0.983) (1.049)
Mother CESD-R score > =10 0.015 0.003 −0.017 1.918 0.456 −0.622 1.315 0.070 −0.082
(0.026) (0.020) (0.022) (2.208) (1.864) (1.827) (1.870) (1.533) (1.671)
Household wealth above median −0.059* −0.036 −0.009 −4.115* −3.623* −1.827 −1.117 −0.799 −0.757
(0.034) (0.029) (0.027) (2.437) (2.152) (2.118) (1.588) (1.473) (1.550)
District & village characteristics
0.086* 4.687 0.629
Distance to district capital above median
(0.051) (3.560) (2.025)
Multivariate regression N N Y N N Y N N Y
District fixed effects N Y Y N Y Y N Y Y
Note: Only children in treatment group. Specifications (7)–(9) include only children with at least one visit. N is 1825 to 3192 depending on the covariate of interest and specification.
Standard errors clustered at district level (in parenthesis).
*Indicate significance at the 10 percent level.
**Indicate significance at the 5 percent level.
***Indicate significance at the 1 percent level.
20
All characteristics refer to data collected at baseline or those that are time-
invariant (child gender, distance to district capital). One exception is child age,
where we use child age at follow-up, as this is what determines whether a child
was still eligible for home visits.
21
In general, we partition the data into values above, or below, the mean of
the corresponding variable. Two exceptions are maternal years of schooling,
where we compare children of mothers who were at least secondary school
graduates with other children, and maternal depression, where we construct
an indicator variable for mothers with CESD-R-10 values above or equal to
Fig. 2. Take-up and dosage. 10, the standard cutoff for self-reported depression (according to the norming
Note: Data from administrative records and survey data. population).
7
M.C. Araujo, M. Dormal, S. Grantham-McGregor et al. Journal of Public Economics Plus 2 (2021) 100003
better-off counterparts. Stunted children and those with below-average received at least one home visit, and in the other, specification (4),
ASQ-3 scores were about 6 percentage points more likely to receive at ‘‘treatment’’ refers to the number of visits received. In both cases, we
least one visit. Maternal vocabulary and the number of assets in the instrument treatment with random assignment.25
home were both associated with fewer visits. Most strikingly, children The sample size is 4,685.26 We report standard errors corrected
of mothers who were high school graduates were 12 percentage points for clustering at the district level, and the corresponding p-values. In
less likely to receive at least one home visit than other children, and addition, for the OLS regressions, we report p-values that correct for
received 7 fewer visits, on average. The last row in the table shows that multiple hypothesis testing using the step-down procedure in Romano
families in villages located further from the district capital were more and Wolf (2005), randomization-inference based p-values (Rosenbaum,
likely to receive home visits.22 2002), and the results of a randomization-based joint significance test,
We do not know why better-off households received fewer visits. It following Young (2019).27
is possible that home visitors made a special effort to reach the poorest
households, or that better-off households were more likely to reject the 4.2. Program effects on child development
program. It may also have been easier to hire, or retain, home visitors
in poorer villages (perhaps because the opportunity cost of working as a Estimates of Cuna Mas effects on child development, based on (1)
home visitor was lower).23 Regardless, the results in Table 2 show that above, are in Table 3. We focus our discussion on the specification
home visiting was an effective way of reaching many of the poorest without controls, but note that results from the OLS regressions with
households, even within (generally quite poor) communities. This is controls tend to be more precise.
encouraging, as, in many settings, it is often the case that better-off Averaging across all domains of the ASQ-3, intent-to-treat results
households are the first to receive a program or benefit. indicate that Cuna Mas improved child development by 0.10 standard
deviations. Treatment-on-the-treated estimates in column (3) show that
4. Results children in families that received at least one visit had child develop-
ment scores that were 0.15 standard deviations higher, while those in
4.1. Estimation strategy column (4) show that, for every 10 visits received, child development
improved by 0.023 standard deviations. In the OLS specification with-
Our results on the effects of Cuna Mas are based on regressions of out controls, p-values are 0.049 when we correct for clustering within
the following form: districts, and 0.122 when we correct for randomization inference.
Other rows in the table indicate that Cuna Mas had positive effects
𝑌𝑖ℎ𝑑𝑡 = 𝛼𝑟 +𝛽1 𝑇𝑑 +𝛽2 𝑌𝑖ℎ𝑑𝑡−1 +𝛽3 𝑋𝑖ℎ𝑑𝑡−1 +𝜀𝑖ℎ𝑑𝑡 , (1) on cognition and language, but not on other domains of development.
where Y is a measure of child development or a parenting practice, This is in line with the results of other impact evaluations of small-
and i indexes children, h households, d districts, r trios and t time scale pilots of home visiting programs in developing countries (see
(the baseline or follow-up surveys). The parameter of interest is 𝛽1 , Grantham-McGregor and Smith, 2016 for a review). Intent-to-treat
which measures whether there are differences between children who effects are 0.10 standard deviations for cognition, and 0.11 standard de-
were randomly assigned to treatment and control groups. viations for language. Impacts are significant or borderline significant
We report the results from two intent-to-treat specifications. Speci- even with corrections for multiple hypothesis testing and randomiza-
fication (1) includes only the fixed effects for trios, 𝛼r ; in specification tion inference, with p-values that range from 0.011 to 0.062. The
(2), we add the controls in Table 2.24 We also report the results p-value on a test of complete irrelevance, following Young (2019),
from two treatment-on-the-treated specifications. In the first of these, is 0.056. The instrumental variables regressions in column (4) of the
specification (3), ‘‘treatment’’ takes on the value of one if a family table show that every 10 home visits improved cognitive development
by 0.022 standard deviations, and language development by 0.025
22
standard deviations.
Information on travel time to the district capital was provided by a
One concern with the results in Table 3 is that the ASQ-3 collects
village leader, in a separate survey. The regressions in the last row of Table 2
some items by direct observation and others by maternal report. This
are based on travel time, in minutes. The respondent also indicated whether
traveling to the district capital was generally done by foot, car, bus, motorbike,
raises the possibility that the Cuna Mas effect we estimate could, in
or boat. We get very similar results to those we report in Table 2 if we part, be driven by reporting or desirability bias—mothers who received
separately calculate above- and below-average travel time for each mode of home visits may have been more likely to report that their child could
transportation, and then partition the sample accordingly. perform a task (like saying a given word) than those in the control
23 group, even if there were no actual differences in child outcomes. To
We control for district fixed effects because random assignment occurred
at the district level. However, for the regressions in Table 2, we can also in- see whether this is likely to be an issue, we make use of the fact that
clude village, rather than district, fixed effects. When we do so, the magnitude one of the ASQ-3 domains, language, includes some items that were
of the coefficients on different measures of socioeconomic status goes down collected by maternal report and others that were directly observed.
substantially. For example, in the regression for the number of visits received, Specifically, we construct two separate language aggregates – one for
the coefficient on the indicator for mothers with at least completed secondary
mother-reported items, the other for directly observed items – and run
education is −7.4 (2.2) without fixed effects, −6.8 (1.8) with district fixed
regressions separately for these two language aggregates. We report
effects, and −4.2 (1.5) with village fixed effects. In the case of households
with above-average assets, the coefficient is −4.1 (2.4) without fixed effects,
and −3.6 (2.2) and −0.78 (1.6) in the regressions with district and village 25
The first stage in these regressions is highly significant—a coefficient of
fixed effects, respectively. In other words, between 44 percent (for maternal 0.67, with a standard error of 0.01 when treatment is defined as receiving
education) and 93 percent (for wealth) of the variation in the number of visits at least one visit, and 4.31, with a standard error of 0.09 when treatment is
received is accounted for by differences across, rather than within, villages. defined as the number of visits a family received.
26
Differences in the number of visits across villages could occur for a number We are missing data on one or more of the variables in Table 1 for 14.2
of reasons, including problems attracting or retaining home visitors, or other percent of children in the sample. In these cases, we replace the missing value
implementation challenges. with the median for the sample and include indicator variables for children
24
We parametrize controls as indicator variables, as in Table 2, and as for whom these variables are missing.
27
recommended in Athey and Imbens (2017), with one exception—we include All of these are available as ado files in Stata. For the Romano–
the lagged value of the ASQ-3 (or of the individual domain) as a third- Wolf correction, see the rwolf command, programmed by Clarke (2016); for
order polynomial of the continuous score. However, results are very similar randomization-based inference, see the ritest command, programmed by Hess
if, instead, we include the indicator variable for children with above-average (2019); and for the joint insignificance test, see randcmd, programmed by
ASQ-3 scores, as we do with the other controls. Young (2020).
8
M.C. Araujo, M. Dormal, S. Grantham-McGregor et al. Journal of Public Economics Plus 2 (2021) 100003
Table 3
Cuna Mas effects on child development.
Intent-to-treat Treatment-on-the-treated
(1) (2) (3) (4)
0.101 0.109 0.151 0.023
(0.051) (0.045) (0.077) (0.012)
All domains
[0.049] [0.017] [0.052] [0.052]
⟨0.122⟩ ⟨0.063⟩
0.095 0.095 0.142 0.022
(0.040) (0.036) (0.059) (0.009)
Cognitive development [0.018] [0.008] [0.018] [0.019]
{0.056} {0.029}
⟨0.051⟩ ⟨0.027⟩
0.109 0.111 0.162 0.025
(0.043) (0.039) (0.064) (0.010)
Language development [0.011] [0.005] [0.012] [0.012]
{0.038} {0.023}
⟨0.049⟩ ⟨0.036⟩
0.068 0.081 0.102 0.016
(0.052) (0.051) (0.079) (0.012)
Personal-social development [0.193] [0.116] [0.200] [0.201]
{0.363} {0.277}
⟨0.325⟩ ⟨0.250⟩
0.068 0.068 0.101 0.016
(0.046) (0.041) (0.069) (0.011)
Fine motor development [0.141] [0.099] [0.144] [0.144]
{0.363} {0.277}
⟨0.213⟩ ⟨0.176⟩
−0.021 −0.012 −0.031 −0.005
(0.045) (0.044) (0.067) (0.010)
Gross motor development [0.646] [0.782] [0.648] [0.648]
{0.658} {0.788}
⟨0.715⟩ ⟨0.802⟩
Trio fixed effects X X X X
SES controls X
Randomization p-value, Westfall–Young 0.056 0.037
joint test
Note: N is 4685 in all specifications. Table reports coefficients, standard errors (in parentheses), p-values [in
square brackets], Romano–Wolf p-values {in curly brackets}, and randomization inference-based p-values <
in angle brackets > from regressions of the dependent variable in the first column on treatment. Specifications
(1) and (2) are intent-to-treat and results refer to children randomly assigned to treatment. In the treatment-
on-the-treated specification (3), ‘‘treatment’’ is defined as children who received at least one home visit
and is instrumented with random assignment to treatment. Similarly, in Specification (4), ‘‘treatment’’ is
defined as the total number of visits received/10 and is instrumented with random assignment to treatment.
Specifications (1), (3) and (4) include only the trio fixed effects. Specification (2) adds the following child
and household baseline controls: child age above 36, gender, a third-order polynomial in baseline ASQ-3
total score, stunting, maternal education is high school or above, maternal age above the median, maternal
TVIP score above the median, mother is depressed (CESD-R-10>= 10), household wealth above the median,
distance to district capital above the median. All standard errors are clustered at district level.
the results from these regressions in Table 4. The table shows that, parenting.28 All responses refer to the week prior to the survey. We
if anything, the estimated effects of home visits are larger for items note that the parenting behaviors we measure are strongly associated
collected by direct observation (0.13 standard deviations) than for with child development in many settings.29 We also have data on the
those collected by maternal report (0.07 standard deviations). We take number of play materials (for example, coloring books or building toys)
this as strong evidence that reporting or desirability biases are not available to the child in the home.
behind the Cuna Mas effects on child development we estimate, at least We parametrize parenting behaviors in two ways. First, we create
for language. indicator variables that take on the value of one if the caregiver indi-
Next, we test for possible heterogeneity of program effects, supple- cated that at least one of the activities in a given category occurred—for
menting equation (1) with an interaction between a given covariate and example, under harsh corporal punishment, whether the child was hit,
treatment, one at a time. Table 5 reports the coefficients on treatment, or spanked, or pinched at least once in the last week. Second, we create
the covariate in question (say, whether a child was older than 36 variables that correspond to the number of activities within a category
months at follow-up, or gender), and the relevant interaction. The table
shows that there is no evidence of significant heterogeneity along any 28
There are 7 questions about play activities (including, for example,
dimension we measure, although the standard errors on the interaction whether someone read to the child, or played with the child), 3 questions
effects are generally quite large. about harsh corporal punishment (including, for example, whether the child
was spanked, or hit with a solid object like a belt), 2 questions about harsh
verbal punishment (including whether someone yelled at or insulted the child),
4.3. Program effects on parenting practices
and 4 questions about positive parenting (including, for example, whether the
caregiver praised or hugged the child).
We analyze Cuna Mas effects on parenting behaviors. For this 29
References include Bradley and Corwyn (2005), Britto et al. (2017), Engle
purpose, we use caregiver responses to questions about play activities, et al. (2011), Galasso et al. (2019), Hamadani et al. (2010), and Hamadani
harsh corporal punishment, harsh verbal punishment, and positive et al. (2014).
9
M.C. Araujo, M. Dormal, S. Grantham-McGregor et al. Journal of Public Economics Plus 2 (2021) 100003
Table 4
Cuna Mas effects on language development, mother-reported versus direct observation.
Intent-to-treat Treatment-on-the-treated
(1) (2) (3) (4)
0.109 0.111 0.162 0.025
(0.042) (0.039) (0.064) (0.010)
Language development (all items) [0.011] [0.005] [0.012] [0.012]
{0.038} {0.023}
⟨0.049⟩ ⟨0.036⟩
0.131 0.129 0.196 0.03
(0.046) (0.043) (0.069) (0.011)
Language development (observation) [0.005] [0.003] [0.005] [0.005]
{0.028} {0.014}
⟨0.044⟩ ⟨0.027⟩
0.065 0.068 0.097 0.015
(0.032) (0.031) (0.048) (0.007)
Language development (maternal report) [0.041] [0.029] [0.045] [0.045]
{0.139} {0.100}
⟨0.130⟩ ⟨0.108⟩
Trio fixed effects X X X X
SES controls X
Randomization p-value, Westfall–Young 0.039 0.016
joint test
Note: N is 4685 in all specifications. Table reports coefficients, standard errors (in parentheses), p-values [in square
brackets], Romano–Wolf p-values {in curly brackets}, and randomization inference-based p-values < in angle brackets
> from regressions of the dependent variable in the first column on treatment. Specifications (1) and (2) are intent-
to-treat and results refer to children randomly assigned to treatment. In the treatment-on-the-treated specification (3),
‘‘treatment’’ is defined as children who received at least one home visit and is instrumented with random assignment
to treatment. Similarly, in Specification (4), ‘‘treatment’’ is defined as the total number of visits received/10 and is
instrumented with random assignment to treatment. Specifications (1), (3) and (4) include only the trio fixed effects.
Specification (2) adds the following child and household baseline controls: child age above 36, gender, a third-order
polynomial in baseline ASQ-3 total score, stunting, maternal education is high school or above, maternal age above
the median, maternal TVIP score above the median, mother is depressed (CESD-R-10>= 10), household wealth above
the median, distance to district capital above the median. All standard errors are clustered at district level.
Table 5
Heterogeneity of Cuna Mas effects.
Age > 36 Gender Baseline Stunted High school Mother TVIP Mother age Mother Household Distance to
months ASQ-3 above complete or above median above CESD-R > wealth above district capital
median above median =10 median above median
Covariate −0.006 0.180*** 0.405*** −0.351*** 0.458*** 0.303*** 0.036 −0.026 0.333*** −0.180**
(0.060) (0.049) (0.063) (0.062) (0.073) (0.056) (0.050) (0.067) (0.063) (0.083)
Treatment 0.027 0.138** 0.122* 0.100* 0.071 0.096 0.100 0.096* 0.054 0.125
(0.073) (0.061) (0.062) (0.055) (0.056) (0.063) (0.064) (0.053) (0.060) (0.081)
Covariate*Treatment 0.107 −0.073 −0.001 0.005 0.060 −0.007 0.001 0.044 0.070 −0.041
(0.074) (0.060) (0.073) (0.076) (0.089) (0.071) (0.064) (0.084) (0.081) (0.105)
Trio fixed effects X X X X X X X X X
Note: N is 4685 in all specifications. Table reports coefficients and standard errors clustered at district level (in parentheses) from regressions of the total ASQ-3 on each covariate,
treatment, and the interaction term. Specifications include trio fixed effects (and no other covariate), except for regressions that correspond to distance to district capital.
*Indicate significance at the 10 percent level.
**Indicate significance at the 5 percent level.
***Indicate significance at the 1 percent level.
that were carried out in the last week—for example, a child who was hit or randomization inference (p-values < 0.01). On the other hand, we
and spanked in the last week would get a value of 2, whereas a child do not see increases in positive parenting or in the number of play
who was only spanked would get a value of 1 on this measure. For materials in the home.
simplicity, we refer to these two measures as the extensive and intensive
Finally, Table 6 shows that among children 3 years of age or older,
margins of parenting behaviors, respectively.
those randomly assigned to the treatment group were 8 percentage
Table 6 shows there were substantial changes in some parenting
behaviors, in particular with regard to how children were disciplined. points more likely to be enrolled in preschool, from a proportion of
Children in the treatment group were less likely to be exposed to 50 percent in the control group. This is a large effect and may be
harsh corporal punishment (a reduction of 7 percentage points, from particularly important given the literature from both developed and
a proportion of 35 percent in the control group), or harsh verbal developing countries, including in Latin America, showing that there
punishment (a reduction of 6 percentage points, from a proportion are substantial, causal effects of preschool attendance on test scores and
of 72 percent in the control group).30 In play activities, there were other outcomes, including in adulthood.31
also changes on the intensive margin. These effects are all significant
even after we account for multiple hypothesis testing (p-values < 0.01)
31
Reviews of the U.S. literature on preschool include Cascio (2009), Cascio
30
There are no increases on the extensive margin in play activities or and Schanzenbach (2013), Cunha and Heckman (2010), Duncan and Magnu-
positive parenting practices, but we note that levels in the control group in son (2013), and Ludwig and Phillips (2007). On Latin America, see Berlinski
these behaviors are very high: 90 percent and 93 percent, respectively. et al. (2009, 2008), and Bastos et al. (2017)
10
M.C. Araujo, M. Dormal, S. Grantham-McGregor et al. Journal of Public Economics Plus 2 (2021) 100003
Table 6
Cuna Mas effects on parenting behaviors.
Dependent variable: Behavior occurred at least once Dependent variable: Number of behaviors
Proportion Intent-to-treat Treatment-on-the-treated Average # Intent-to-treat Treatment-on-the-treated
(control of items
group) [range]
(control
group)
(1) (2) (3) (4) (5) (6) (7) (8)
0.012 0.013 0.019 0.003 0.366 0.379 0.545 0.085
0.9 (0.018) (0.017) (0.027) (0.004) 3.47 (0.147) (0.125) (0.219) (0.034)
Play activities [0.495] [0.447] [0.497] [0.496] [0-7] [0.014] [0.003] [0.014] [0.014]
{0.586} {0.585} {0.039} {0.009}
⟨0.555⟩ ⟨0.5327⟩ ⟨0.044⟩ ⟨0.020⟩
−0.072 −0.072 −0.107 −0.017 −0.105 −0.104 −0.157 −0.024
0.35 (0.018) (0.018) (0.028) (0.004) 0.43 (0.028) (0.028) (0.042) (0.007)
Harsh corporal punishment [0.000] [0.000] [0.000] [0.000] [0-3] [0.000] [0.000] [0.000] [0.000]
{0.000} {0.000} {0.001} {0.002}
⟨0.001⟩ ⟨0.001⟩ ⟨0.002⟩ ⟨0.002⟩
−0.064 −0.065 −0.096 −0.015 −0.086 −0.089 −0.128 −0.02
0.72 (0.019) (0.017) (0.028) (0.004) 0.98 (0.033) (0.031) (0.050) (0.008)
Harsh verbal punishment [0.001] [0.000] [0.001] [0.001] [0-2] [0.010] [0.004] [0.011] [0.011]
{0.000} {0.000} {0.039} {0.009}
⟨0.007⟩ ⟨0.002⟩ ⟨0.036⟩ ⟨0.016⟩
0.008 0.008 0.013 0.002 0.021 0.019 0.032 0.005
0.93 (0.009) (0.009) (0.014) (0.002) 1.23 (0.042) (0.039) (0.062) (0.010)
Positive parenting [0.368] [0.362] [0.368] [0.368] [0-4] [0.612] [0.627] [0.612] [0.612]
{0.586} {0.586} {0.829} {0.856}
⟨0.452⟩ ⟨0.442⟩ ⟨0.676⟩ ⟨0.681⟩
0.017 0.015 0.025 0.004 0.041 0.021 0.061 0.009
0.94 (0.012) (0.012) (0.018) (0.003) 3.56 (0.169) (0.140) (0.252) (0.039)
Play materials [0.178] [0.205] [0.181] [0.180] [0-8] [0.809] [0.880] [0.809] [0.809]
{0.428} {0.474} {0.829} {0.869}
⟨0.271⟩ ⟨0.289⟩ ⟨0.842⟩ ⟨0.894⟩
0.083 0.081 0.128 0.021
0.50 (0.023) (0.023) (0.036) (0.006)
Preschool (>= 36
[0.000] [0.001] [0.000] [0.000]
months at follow-up)
{0.004} {0.003}
⟨0.003⟩ ⟨0.002⟩
Trio fixed effects X X X X X X X X
SES controls X X
Randomization p-value, 0.004 0.002 0.001 0.001
Westfall–Young joint test
Note: N is 4685 in all specifications, except for play materials (N = 4625) and preschool attendance (N = 3422). Table reports coefficients, standard errors (in parentheses), p-values
[in square brackets], Romano–Wolf p-values {in curly brackets}, and randomization inference-based p-values < in angle brackets > from regressions of the dependent variable in
the first column on treatment. Specifications (1), (2), (5) and (6) are intent-to-treat and results refer to children randomly assigned to treatment. In the treatment-on-the-treated
specifications (3) and (7), ‘‘treatment’’ is defined as children who received at least one home visit and is instrumented with random assignment to treatment. Similarly, in the
specifications (4) and (8), ‘‘treatment’’ is defined as the total number of visits received/10 and is instrumented with random assignment to treatment. All specifications include only
the trio fixed effects, except for specifications (2) and (6), which add the following child and household baseline controls: child age above 36, gender, a third-order polynomial
in baseline ASQ-3 total score, stunting, maternal education is high school or above, maternal age above the median, maternal TVIP score above the median, mother is depressed
(CESD-R-10>= 10), household wealth above the median, distance to district capital above the median. All standard errors are clustered at district level.
4.4. Benefit–cost ratios were eligible for 24 months. We also note that, in Colombia, virtually
all children (97.2 percent) assigned to treatment received at least one
Should we think of the Cuna Mas impacts on child development visit. As discussed above, that was not the case in Peru. We assume
as ‘‘small’’ or ‘‘large’’? As a first approximation to this question, we that the program costs were approximately zero for families that did
compare the magnitude of the Cuna Mas impacts to the associations not receive a single visit in Peru, so in Peru we use the treatment-
between maternal education and child development at follow-up for on-the-treated estimates of program impact. Finally, we note that, in
children in the control group. In a simple regression of the average Colombia, the measure of child development included only cognitive
ASQ-3 score on the years of completed schooling of mothers, without development, language, and fine motor development. Therefore, to
any controls, the coefficient is 0.06 (with a standard error of 0.01). make outcomes comparable across the two countries, we construct an
By this measure, the treatment-on-the-treated effects we estimate are aggregate that only considers these three domains of the ASQ-3. The
equivalent to an increase of 2.5 years of maternal schooling, in a estimated treatment-on-the-treated Cuna Mas effect on this measure,
population in which mean schooling was 7 years. without covariates, is 0.19 standard deviations.
Next, we calculate the benefit–cost ratios of Cuna Mas, and compare To calculate the monetary value of these program effects, we turn
these estimates with those for the Colombian pilot.32 To calculate to the data from Jamaica. The impact of home visiting on an overall
program benefits, we proceed as follows. We first note that children in measure of child development after 24 months in Jamaica was 0.88
Colombia received visits for 18 months, on average, while those in Peru standard deviations. In Peru, the comparable effect was 0.19 standard
deviations and, in Colombia, after adjusting for the fact that the pilot
32
We cannot calculate benefit–cost ratios for the Jamaica efficacy trial, as lasted for only 18 months, it was 0.24 standard deviations. Thus, the
we do not know the costs of home visits there. short-term effects of home visits in Colombia and Peru were 27 percent
11
M.C. Araujo, M. Dormal, S. Grantham-McGregor et al. Journal of Public Economics Plus 2 (2021) 100003
(0.24/0.88) and 21 percent (0.19/0.88), respectively, of those observed Declaration of competing interest
in Jamaica. In Jamaica, the short-term effects on child development led
to a 25 percent increase in labor income. We therefore estimate that, The authors declare that they have no known competing finan-
for those who work, the expected increase in labor income would be 6.8 cial interests or personal relationships that could have appeared to
percent in Colombia (0.27*0.25), and 5.3 percent (0.21*0.25 in Peru).33 influence the work reported in this paper.
We use labor force surveys in Colombia and Peru to calculate the
Net Present Value (NPV) of the expected increase in future earnings Appendix A and B. Supplementary data
for individuals who received home visits.34 In carrying out these calcu-
Supplementary material related to this article can be found online
lations, we assume that people can earn labor income when they are
at https://doi.org/10.1016/j.pubecp.2021.100003.
between 25 and 65 years of age. Conservatively, we also assume that
the benefits of the home visiting intervention apply only to those who
References
work, so we adjust the increase in labor income by the participation
rate at every year of age. We use a 3 percent discount rate, and estimate Aboud, Frances E., Yousafzai, Aisha K., 2015. Global health and development in early
that the NPV of the lifetime increase in earnings per participant is US childhood. Ann. Rev. Psychol. 66, 433–457.
$3,418 in Colombia and US $3,210 in Peru. Finally, we use the data on Al-Ubaydli, Omar, List, John A., Suskind, Dana L., 2017. What can we learn from
experiments? Understanding the threats to the scalability of experimental results.
annual costs per child, which are estimated to be US $500 in Colombia Amer. Econ. Rev. 107 (5), 282–286.
and US $300 in Peru. Combining the data on benefits and costs yields Almond, Douglas, Currie, Janet, 2011. Human capital development before age five. In:
a benefit–cost ratio of 5.4 in Peru and 4.6 in Colombia. Handbook of Labor Economics, Vol. 4. Elsevier, pp. 1315–1486.
Almond, Douglas, Currie, Janet, Duque, Valentina, 2018. Childhood circumstances and
adult outcomes: Act II. J. Econ. Lit. 56 (4), 1360–1444.
5. Conclusion Andrew, Alison, Attanasio, Orazio, Fitzsimons, Emla, Grantham-McGregor, Sally,
Meghir, Costas, Rubio-Codina, Marta, 2018. Impacts 2 years after a scalable early
childhood development intervention to increase psychosocial stimulation in the
Interventions in early childhood can have large effects but, with the home: A follow-up of a cluster randomised controlled trial in Colombia. PLoS Med.
15 (4), e1002556. http://dx.doi.org/10.1371/journal.pmed.1002556.
exception of a rich literature on Head Start in the U.S., most of the
Araujo, M. Caridad, Dormal, Marta, Schady, Norbert, 2019. Child care quality and child
evidence comes from efficacy trials or pilots of carefully administered development. J. Hum. Resour. 54 (3), 656–682.
programs. As is well understood, there are a variety of reasons why the Araujo, M. Caridad, Rubio-Codina, Marta, Schady, Norbert, 2021. 70 to 700 to 70,000:
impacts found in these pilots may not translate into comparable effects lessons from the jamaica experiment. In: List, J., Suskind, D., Supplee, L. (Eds.),
The Scale-Up Effect in Early Childhood & Public Policy: Why Interventions Lose
when interventions are taken to scale (List et al., 2021).
Impact at Scale and What We Can Do About It. Routledge.
In this paper, we use a randomized control trial to estimate the im- Athey, Susan, Imbens, Guido W., 2017. The econometrics of randomized experi-
pacts of a home visiting program which, at the time of the evaluation, ments. In: Banerjee, Abhijit, Duflo, Esther (Eds.), Handbook of Economic Field
Experiments, Vol. 1. Elsevier, pp. 73–140.
reached 67,000 families. We show that children who received home
Attanasio, Orazio, 2015. The determinants of human capital formation during the early
visits had development outcomes that were 0.15 standard deviations years of life: Theory, measurement, and policies. J. Eur. Econom. Assoc. 13 (6),
higher than those who did not receive them. The estimated benefit–cost 949–997.
ratio of the program is substantial, 5.4. Children randomly assigned to Attanasio, Orazio, Fernández, Camila, Fitzsimons, Emla, Grantham-McGregor, Sally,
Meghir, Costas, Rubio-Codina, Marta, 2014. Using the infrastructure of a con-
treatment were also less likely to be exposed to harsh corporal or verbal
ditional cash transfer program to deliver a scalable integrated early child
punishment, and were more likely to be enrolled in preschool upon development program in Colombia: Cluster randomized controlled trial. Br. Med.
graduation from Cuna Mas. We stress that these positive effects refer J. 349, g5785.
to a program that was started from scratch and went to scale in a very Bando, Rosangela, López-Bóo, Florencia, Li, Xia, 2016. Sex-differences in language
and personal-social skills in early childhood. In: IDB-WP-714. Inter-American
short period of time, with all the attendant implementation difficulties. Development Bank, Washington, D.C..
Will the effects of Cuna Mas be long-lasting? It is of course too Banerjee, Abhijit, Banerji, Rukmini, Berry, James, Duflo, Esther, Kannan, Harini,
early to tell. Results from Jamaica, like those from model programs Mukerji, Shobhini, Shotland, Marc, Walton, Michael, 2017. From proof of concept
to scalable policies: Challenges and solutions, with an application. J. Econ. Perspect.
implemented on a small scale in the U.S., are striking.35 On the other
31 (4), 73–102.
hand, the home visiting program in Colombia, which had short-term Bastos, Paulo, Bottan, Nicolas, Cristia, Julian, 2017. Access to preprimary education
effects that are more similar in magnitude to those we estimate in Peru, and progression in primary school: Evidence from rural Guatemala. Econom. Dev.
are less encouraging. Our results point to the importance of following Cult. Chang. 65 (3), 521–547.
Berlinski, Samuel, Galiani, Sebastian, Gertler, Paul, 2009. The effect of pre-primary
children like those in the Cuna Mas evaluation through their schooling education on primary school performance. J. Public Econ. 93 (1), 219–234.
career and, eventually, into adulthood. Berlinski, Samuel, Galiani, Sebastian, Manacorda, Marco, 2008. Giving children a better
start: Preschool attendance and school-age profiles. J. Public Econ. 92 (5–6),
1416–1440.
33 Björgvinsson, Thröstur, Kertz, Sarah J., Bigda-Peyton, Joe S., McCoy, Katrina L.,
The expected increase in labor income in Colombia and Peru should be
seen as suggestive only, for many reasons. First, the measure of child devel- Aderka, Idan M., 2013. Psychometric properties of the CES D-10 in a psychiatric
opment was not the same across countries (the Griffiths Mental Development sample. Assessment 20 (4), 429–436.
Björkman Nyqvist, Martina, Guariso, Andrea, Svensson, Jakob, Yanagizawa-
Scales in Jamaica, the Bayley Scales for Infant and Toddler Development in
Drott, David, 2019. Reducing child mortality in the last mile: Experimental
Colombia, and the ASQ-3 in Peru). Second, our estimates assume that, if the evidence on community health promoters in Uganda. Am. Econ. J.: Appl. Econ.
impact of a program on child development in country A (Colombia or Peru) 11 (3), 155–192.
is a proportion x of the impact of a similar program in country B (Jamaica), Bradley, Robert, Corwyn, Robert, 2005. Caring for children around the world: A view
then the future impact on wages in country A will also be a proportion x of from HOME. Int. J. Behav. Dev. 29 (6), 468–478.
the impact on wages observed in country B. In practice, the mapping from Britto, Pia R., Lye, Stephen L., Proulx, Kerrie, Yousafzai, Aisha K., Matthews, Stephen G.,
changes in development to changes in wages may be less straightforward—for Vaivada, Tyler, et al., 2017. Nurturing care: Promoting early childhood
example, it could be nonlinear, or it could depend on other factors that vary development. Lancet 389 (10064), 91–102.
Caldwell, Bettye M., Bradley, Robert H., 2001. HOME Administration Manual, third ed.
by country, such as availability and quality of education services.
34 University of Arkansas University, Little Rock, Print Design.
We used the nationwide surveys that were closest to the date of the
Cascio, Elizabeth, 2009. Do investments in universal education pay off? long-term
evaluation—namely, in Colombia, the 2011 Gran Encuesta Integrada de Hogares effects of introducing kindergartens into public schools. NBER Working Paper.
(GEIH), and in Peru, the 2018 Encuesta Nacional de Hogares (ENAHO). 14951.
35
See Heckman et al. (2010) on the Perry Preschool Program, and García Cascio, Elizabeth, Schanzenbach, Diane Whitmore, 2013. The impacts of expending
et al. (2020) on the Carolina Abecedarian Project. access to high-quality preschool education. NBER Working Paper. 19735.
12
M.C. Araujo, M. Dormal, S. Grantham-McGregor et al. Journal of Public Economics Plus 2 (2021) 100003
Chapman, Donna J., Morel, Katherine, Anderson, Alex Kojo, Damio, Grace, Perez- Macours, Karen, Schady, Norbert, Vakis, Renos, 2012. Cash transfers, behavioral
Escamilla, Rafael, 2010. Breastfeeding peer counselling: From efficacy to scale-up. changes, and cognitive development in early childhood: Evidence from a
J. Hum. Lact. 26 (3), 314–326. randomized experiment. Am. Econ. J.: Appl. Econ. 4 (2), 247–273.
Clarke, Damien, 2016. RWOLF: Stata module to calculate Romano-Wolf stepdown p- Mayoral, Maria V., 2014. Fact sheet: Early Head Start. Zero to three national center for
values for multiple hypothesis testing. In: Statistical Software Components S458276. infants, toddlers, and families. Available at: http://www.buildinitiative.org/Portals/
Boston College Department of Economics. 0/Uploads/Documents/Fact%20Sheet%20-%20Early%20Head%20Start%20-
Cunha, Flavio, Heckman, James, 2010. Investing in our young people. NBER Working %20ZTT.pdf. Consulted in April 2019.
Paper. 16201. Olds, David L., 2010. The Nurse-Family Partnership: From trials to practice. In:
DGSE-MIDIS, DGPP-MEF, BID, 2015. Documento de línea de base para la evaluación de Reynolds, Arthur, Rolnick, Arthur, Englund, Michelle, Temple, Judy (Eds.), Child-
impacto del programa nacional Cuna Más. Servicio de acompañamiento a familias, hood programs and practices in the first decade of life: A human capital integration.
Lima, febrero de, Documento del MIDIS. Available at: http://www.midis.gob. Cambridge University Press, Cambridge, pp. 49–75.
pe/dgsye/evaluacion/documentos/CUNAMAS-AF_30_de_Enero.pdf. (Accessed on 23 Paxson, Christina, Schady, Norbert, 2010. Does money matter? The effects of cash
March 2017). transfers on child health and development in rural Ecuador. Econom. Dev. Cult.
Duncan, Greg J., Magnuson, Katherine, 2013. Investing in preschool programs. J. Econ. Chang. 59 (1), 187–229.
Perspect. 27 (2), 109–132. Prost, Audrey, Colbourn, Tim, Seward, Nadine, Azad, Kishwar, Coomarasamy, Arri,
Dunn, Lloyd, Lugo, Delia, Padilla, Eligio, Dunn, Leota, 1986. Test de Vocabulario En Copas, Andrew, et al., 2013. Women’s groups practising participatory learning
Imágenes Peabody. American Guidance Service, Circle Pines, MN. and action to improve maternal and newborn health in low-resource settings: A
Engle, Patrice, Fernald, Lia C.H., Alderman, Harold, Behrman, Jere, O’Gara, Chloe, systematic review and meta-analysis. Lancet 381 (9879), 1736–1746.
Yousafzai, Aisha, et al., 2011. Strategies for reducing inequalities and improving Radloff, Lenore Sawyer, 1997. The CES D scale: A self-report depression scale for
developmental outcomes for young children in low-income and middle-income research in the general population. Appl. Psychol. Meas. 1 (3), 385–401.
countries. Lancet 378 (9799), 1339–1353. Rollins, Nigel C., Bhandari, Nita, Hajeebhoy, Nemat, Horton, Susan, Lutte, Chessa,
Galasso, Emmanuela, Weber, Ann, Fernald, Lia C.H., 2019. Dynamics of child develop- Martines, Jose, et al., 2016. Why invest, and what it will take to improve
ment: Analysis of a longitudinal cohort in a very low income country. World Bank breastfeeding practices? Lancet 387 (10017), 491–504.
Econ. Rev. 33 (1), 140–159. Romano, Joseph P., Wolf, Michael, 2005. Stepwise multiple testing as formalized data
García, Jorge Luis, Heckman, James J., Leaf, Duncan Ermini, Prados, María José, 2020. snooping. Econometrica 73 (4), 1237–1282.
Quantifying the life-cycle benefits of an influential early childhood program. J. Rosenbaum, Paul R., 2002. Observational studies, second ed. Springer, New York.
Polit. Econ. 128 (7), 2502–2541. Rubio-Codina, Marta, Araujo, M. Caridad, Attanasio, Orazio, Muñoz, Pablo, Grantham-
Gertler, Paul, Heckman, James, Pinto, Rodrigo, Zanolini, Adriana, Vermeersch, Christel, McGregor, Sally, 2016a. Concurrent validity and feasibility of short tests currently
Walker, Susan, Chang, Susan, Grantham-McGregor, Sally, 2014. Labor market used to measure early childhood development in large scale studies. PLoS One 11
returns to an early childhood stimulation intervention in Jamaica. Science 344 (8), e0160962.
(6187), 998–1001. Rubio-Codina, Marta, Tomé, Romina, Araujo, M. Caridad, 2016b. Una fotografía
Grantham-McGregor, Sally M., Powell, Christine A., Walker, Susan P., Himes, John H., sobre el bienestar y el desarrollo de los niños del programa nacional Cuna Más.
1991. Nutritional supplementation, psychosocial stimulation, and mental de- IDB-TN-1093. Inter-American Development Bank, Washington, D.C.
velopment of stunted children: the Jamaican Study. Lancet 338 (8758), Salam, Rehana, Das, Jai K., Lassi, Zohra, Bhutta, Zulfiqar, 2014. Impact of community-
1–5. based interventions for the prevention and control of malaria on intervention
Grantham-McGregor, Sally, Smith, Joanne A., 2016. Extending the Jamaican early coverage and health outcomes for the prevention and control of malaria. Infect.
childhood development intervention. J. Appl. Res. Child.: Inf. Policy Child. Risk Dis. Poverty 3 (25), 1–15.
7 (2), 4. Schady, Norbert, 2011. Parental education, vocabulary, and cognitive development in
Hamadani, Jena, Fahmida, Tofail, Huda, Syed N., Alam, Dewan S., Ridout, Deborah A., early childhood: Longitudinal evidence from Ecuador. Am J Public Health 101 (12),
Attanasio, Orazio, Grantham-McGregor, Sally, 2014. Cognitive deficit and poverty 2299–2307.
in the first 5 years of childhood in Bangladesh. Pediatrics 134 (4), e1001–e1008. Sohr-Preston, Sara L., Scaramella, Laura V., 2006. Implications of timing of maternal
Hamadani, Jena, Tofail, Fahmida, Hilaly, Afroza, Huda, Syed N., Engle, Patrice L., depressive symptoms for early cognitive and language development. Clin. Child
Grantham-McGregor, Sally, 2010. Use of family care indicators and their rela- Family Psychol. Rev. 9 (1), 65–83.
tionship with child development in Bangladesh. J. Health Popul. Nutr. 28 (1), Squires, Jane, Bricker, Diane, Twombly, Elizabeth, Nickel, Robert, Clifford, Jantina,
23–33. Murphy, Kimberly, Hoselton, Robert, Potter, Lawanda, Mounts, Linda, Farrell, Jane,
Haroon, Sarah, Das, Jai k., Salam, Rehana A., Imdad, Aamer, Bhutta, Zulfiqar A., 2013. 2009. Ages & Stages English questionnaires. In: (ASQ-3): a parent-completed,
Breastfeeding promotion interventions and breastfeeding practices: A systematic child-monitoring system, third ed. Paul H. Brookes Publishing Co, Baltimore, MD.
review. BMC Public Health 13 (3), 1–18. Walker, Susan P., Chang, Susan M., Smith, Joanne A., Baker-Henningham, Helen,
Heckman, James J., Moon, Seong Hyeok, Pinto, Rodrigo, Savelyev, Peter A., 2018. The Reach Up early childhood parenting program: Origins, content, and
Yavitz, Adam, 2010. The rate of return to the High Scope Perry preschool program. implementation. Zero to Three 38 (4), 37–43.
J. Public Econ. 94 (1–2), 114–128. Walker, Susan P., Chang, Susan M., Vera-Hernández, Marcos, Grantham-
Heckman, James J., Mosso, Stefano, 2014. The economics of human development and McGregor, Sally, 2011. Early childhood stimulation benefits adult competence and
social mobility. Annu. Rev. Econ. 6 (1), 689–733. reduces violent behavior. Pediatrics 127 (5), 849–857.
Heller, Sara B., Shah, Anuj K., Guryan, Jonathan, Ludwig, Jens, Mullainathan, Sendhil, Walker, Susan P., Grantham-McGregor, Sally M., Powell, Christine A., Chang, Susan M.,
Pollack, Harold A., 2017. Thinking, fast and slow? Some field experiments to reduce 2000. Effects of growth restriction in early childhood on growth, IQ, and cognition
crime and dropout in Chicago. Q. J. Econ. 132 (1), 1–54. at age 11 to 12 years and the benefits of nutritional supplementation and
Hess, Simon, 2019. RITEST: Stata module to perform randomization inference and psychosocial stimulation. J. Pediatr. 137 (1), 36–41.
permutation tests. In: Statistical Software Components S458599. Boston College Walker, Susan P., Wachs, Theodore D., Meeks-Gardner, Julie, Lozoff, Betsy, Wasser-
Department of Economics. man, Gail A., Pollitt, Ernesto, Carter, Julie A., International Child Development
Kariger, Patricia, Frongillo, Edward A., Engle, Patricia, Britto, Pia R., Sywulka, Sara M., Steering Group, 2007. Child development: Risk factors for adverse outcomes in
Menon, Purmina, 2012. Indicators of family care for development for use in developing countries. Lancet 369 (9556), 145–157.
multicountry surveys. J. Health Popul. Nutr. 30 (4), 472–486. Young, Alwyn, 2019. Channeling Fisher: Randomization tests and the statistical
List, John, Suskind, Dana, Supplee, Lauren (Eds.), 2021. The scale-up effect in early insignificance of seemingly significant experimental results. Q. J. Econ. 134 (2),
childhood and public policy: why interventions lose impact at scale and what we 557–598.
can do about it. Routledge. Young, Alwyn, 2020. RANDCMD: Stata module to compute randomization inference
Love, John M., Chazan-Cohen, Rachel, Raikes, Helen, Brooks-Gunn, Jeanne, 2013. p-values. In: Statistical Software Components S458774. Boston College Department
What makes a difference: Early Head Start evaluation findings in a developmental of Economics.
context. In: Monographs of the society for research in child development, 78, (1), Zwane, Alix Peterson, Kremer, Michael, 2007. What works in fighting diarrheal diseases
pp. vii–viii. in developing countries? A critical review. World Bank Res. Observer 22 (1), 1–24.
Ludwig, Jens, Phillips, Deborah A., 2007. The benefits and costs of Head Start. NBER
working paper 12973.
13