Introduction

Self-control and attention-regulation in early childhood are highly predictive of school performance [1–4], workplace success [5, 6], health [6–8], and life satisfaction [9–11]. They are often more predictive than IQ [6, 12, 13] or socio-economic status (SES) [6, 14]. Children who enter school with poorer academic skills and poorer self-control and attention-regulation quickly fall behind and the gap progressively increases in school achievement [14, 15] and health [16, 17]. Hence there is great interest in helping children enter Grade 1 with the academic and executive function (EF) skills they need to launch them on a positive trajectory.

Similarly, social-emotional well-being in childhood predicts better school performance [18–21] and better outcomes on diverse variables in adults [19]. Student bullying and peer exclusion are major social and mental health concerns [22, 23] and classroom stress is causing teachers to burn-out and leave the profession in unprecedented numbers [24–26]. Hence there is great interest in improving students’ camaraderie and kindness, and reducing stress, in the classroom.

Much of the focus has been on prekindergarten programs [27–32]. Kindergarten represents a much less-studied context for investigating ways to improve social-emotional and EF competencies. Yet free, public kindergarten is available throughout most developed countries. At least one longitudinal study reports that attending a higher quality kindergarten is associated with higher rates of college attendance and higher earnings in adulthood [33].

This paper reports the results of a study that investigated whether the Tools of the Mind (Tools) kindergarten program could improve self-control and attention-regulation, academic performance, prosocial behavior, and reduce classroom stress and teacher burnout.

Reducing stress and increasing social harmony are not only important as factors that improve EFs and academic performance, but are important goals in their own right. A program that can reduce ostracism and bullying and reduce teacher burnout is one worth taking a look at. Such a program was examined here.

To unpack the terms used above a bit, self-control and attentional control comprise the “inhibitory control” component of EFs [34]. Self-control involves resisting temptations (including all the temptations not to stay on task or see it through to completion) and resisting speaking or acting reflexively (e.g., instead of responding immediately, giving oneself time to think or calm down before acting). Attentional control involves resisting distractions, being able to pay attention and stay focused for an extended period.

These inhibitory control abilities are critical for success in school [1–4] and in social relations [35–37]. They are needed for inhibiting all the pulls not to pay attention or stay focused and also for complying with school norms, such as staying seated or raising one’s hand, and social norms, such as not grabbing what someone else has or not talking while someone else is speaking. This is probably one of the many reasons why EFs, social-emotional competence, and academic performance are highly interrelated, e.g. [38–40].

The other EFs are working memory, cognitive flexibility, reasoning, and planning [34], but it is inhibitory control that is most predictive of long-term outcomes [6, 7]. A reasonable prediction is that a school program that improves inhibitory control, in addition to addressing academic skills, should produce better academic outcomes than programs that address academic skills but do not address inhibitory control or do so less successfully. We tested that prediction here.

If a person feels lonely or rejected, or is stressed or sad, that negatively impacts inhibitory control, academic performance, and physical and mental health (evidence that loneliness impairs EFs and specifically inhibitory control [41–43], academic performance [20, 44], and health [42, 45–47]; evidence that stress impairs EFs and specifically inhibitory control [48–50], academic performance [51–53], and health [54–56]; evidence that prolonged sadness impairs EFs and specifically inhibitory control [57–59], academic performance [60, 61], and health [62–65]. Therefore, a reasonable prediction is that in a school program that promotes students working together and being kind to and supporting one another (i.e., prosocial behavior [66]) one should find less peer rejection, more joy in the classroom, less teacher burnout, and better student academic performance and inhibitory control.

Tools is a kindergarten curriculum that focuses as much on improving EFs (especially inhibitory control), classroom climate, prosocial behavior and interpersonal skills as on improving academic skills. There is already evidence that it improves EFs, academic performance, and teacher-child relationships and reduces aggression [67–69], though when only parts of the program have been implemented as an add-on to the curriculum, those benefits have not been observed [70, 71].

Three independent evaluations of Tools have been published. The first, published in Science [69], found that recent graduates of Tools showed much better attention-regulation on a Flanker-type task (85% vs. 50% correct) than controls. Children were not evaluated before the intervention so it is possible that children in Tools had better attention-regulation at the outset, though the groups were closely matched on many demographic variables. At-risk, low-income children had been randomized to Tools or to another new curriculum that the school district had developed and predicted would outperform Tools. One school became so impressed by how much Tools children were out-performing others that they dropped out of the study and switched all kindergarten classes to Tools, feeling it unethical to deprive any of Tools.

A much larger study [67, 68] found better and more improved vocabulary, math, teacher-reported teacher-child relationships, and emotion-regulation on the dot-probe task. They did not find, however, better or more improved inhibitory control or cognitive flexibility on the Hearts and Flowers task, card sorting, or Flanker tasks in kindergarten children in Tools versus controls. They also found less and more reduced teacher-reported conduct problems or aggression in kindergarten children in Tools versus controls. Academic benefits were even larger the following year (Grade 1), where gains in reading first became evident. Effects were about eight times larger in low-income schools.

The third study [72] compared a daycare-based Tools program for children 3–4 years old to a high-quality, existing play-based program. Children in Tools whose parents rated them as highly hyperactive and/or inattentive in the Fall showed greater gains on an inhibitory control task of self-control than control children. The authors concluded that “Tools may be advantageous in classrooms with children experiencing greater challenges with self-regulation, at no apparent cost to those less challenged in this regard” (p. 2).

We predicted we would find benefits from Tools on important variables not previously investigated: (a) the academic skill of writing, (b) camaraderie and helping one another in the classroom, or its flip side reduced peer ostracism and exclusion, (c) teachers’ joy in teaching, (d) students’ joy in learning, and (e) EFs in the real-world versus on laboratory measures (specifically the ability to inhibit distraction in the classroom and stay on task), in addition to replicating previously demonstrated benefits to reading. We predicted that classrooms with less play, hands-on learning, or incorporation of training and scaffolding of EFs in school activities, even if they spent more time on academic content, would be less successful in improving academic outcomes and would be characterized by greater stress in the classroom and more teacher burnout.

Results

Descriptive statistics

Teachers and students were well matched in the two groups. See Table 1. Most teachers in both groups were outstanding and very experienced. There were nine teachers (schools) per group; 172 children in the Tools group; 180 children in the control group.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Descriptive statistics.

https://doi.org/10.1371/journal.pone.0222447.t001

Reading

At the beginning of kindergarten, most children could not read even the simplest words. Most classes had no child who could read more than the simplest sentences; the exceptions were one Tools class and three control classes which each had three children who could read at a higher level. No significant difference in reading skills was found between Tools and control classes in September.

By May, eight of the nine Tools classes had more than two children reading at Grade 1 level or higher, while only one of the nine control classes had more than two children reading at Grade 1 level or higher. Children in Tools made significantly greater progress in reading than children in the control group (χ²(1, N = 18) = 4.64, p = 0.02, odds ratio = 3.25). Three times more children were reading at Grade 1 level or higher by May in Tools classes than in control classes (33% vs. 10%): F(1,15) = 6.67, p < 0.02, partial eta squared (ηp²) = 0.33. See Fig 1. Conversely, almost three times more children were still non-readers by May in control classes than in Tools classes (28% vs. 10%): F(1,15) = 6.02, p = 0.02, ηp² = 0.29.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Reading skills.

By May, three times as many children in Tools than in control classes were reading at Grade 1 level or better, although both groups started out comparably in the Fall.

https://doi.org/10.1371/journal.pone.0222447.g001

This better progress in reading with Tools was also reflected in comments by teachers and parents (see S5 File–Comments by Teachers, Parents, and Principals). Most Tools teachers said they had never seen progress like this in reading before.: “The literacy level in the classroom this year is much higher [than in past years].”

Lower-income children in Tools (those receiving subsidized lunch at school) did not show greater progress in reading than did children in Tools from more prosperous homes (χ²(1, N = 9) = 4.17, p = 0.12 [NS], odds ratio = 2.05; here the covariate was ESL status instead of subsidized lunch). With only nine Tools classes, though, there was limited power to detect a difference. The reading of those who started farther behind in September, however, showed far more progress than the reading of those who started out reading at a more advanced level, as the regression of the difference in reading level (May minus September) against reading level in September shows: F(2,6) = 18.18, p < 0.005, R² = 0.89.

Writing

Children in the two groups started out similarly in writing ability (χ²(1, N = 18) = 1.45, p > 0.20 [NS], odds ratio = 1.10). In September, roughly three children per class in both groups (range = 1–5) could do no better than scribble. In 67% of Tools classes and 56% of control classes, most children could write their first name without copying (85% of children in Tools and 87% of control children). By May, almost all children in both groups could do better than that. The difference was in how far they had progressed. Children in Tools progressed much farther (χ²(1, N = 18) = 20.20, p < 0.001, odds ratio = 26.18). Three times as many children in Tools versus control classes reached as far as being able to write a full sentence they themselves composed with most sounds represented (30% vs. 10%). Almost three times as many children in Tools versus control classes progressed further than that; they could write 2 or more consecutive sentences they composed with most sounds represented (33% vs. 12%). More children in Tools than in control classes progressed from September to May to being able to write a full sentence or multiple consecutive ones that themselves composed with most sounds represented: F(1,15) = 18.10, p < 0.005, ηp² = 0.55. See Fig 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Writing skills.

By May, three times more children in Tools than in control classes were able to write a full sentence they themselves composed or multiple consecutive ones (F[1,15] = 18.10, p < 0.001, ηp² = 0.55).

https://doi.org/10.1371/journal.pone.0222447.g002

It is not so surprising that the writing of children in Tools advanced further than the writing of control children since Tools emphasized writing and control classes did not, though the advanced level of writing by children in Tools would astonish most kindergarten teachers. Indeed, we had to add questions about children’s writing skills to the online teacher survey because the writing levels achieved by children in Tools exceeded the upper limits on the BC assessment tools for kindergarten. Teachers reported never having seen writing progress like this before (see S5 File) and the data bear them out (see Fig 3).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Percentage of children able to write at least one sentence they composed, comparing outcomes for children taught by the teachers assigned to Tools in the pre-Tools year (before Tools was implemented) and Year 1 of Tools.

Children taught by the same teachers the year before those teachers implemented Tools were significantly less advanced in writing than children taught by those teachers the next year when they were implementing Tools: χ²(1, N = 8) = 13.54, p < 0.01, odds ratio = 9.42).

https://doi.org/10.1371/journal.pone.0222447.g003

Social inclusion and other prosocial behavior

Both groups started off comparably. Tools teachers reported that in the Fall they had 3–8 children who had difficulty interacting in the classroom (mean (SD) = 5 (0.7) per class; 26%). Control teachers reported that they had 0–9 children who had difficulty interacting in the classroom (mean (SD) = 4 (2.5) per class; 20%). By May, the percentage of children reported to be having problems interacting was lower in Tools than in control classes (F[1,15] = 6.83, p < 0.02, ηp² = 0.31) and it had gone down much more in Tools than in control classes (F[1,15] = 20.59, p < 0.001, ηp² = 0.58).

Tools teachers commented, for example: “In years past, they have not helped each other to this degree. This year I have witnessed many students going to another student’s aid.” “They offer help and assistance when needed without being asked and without belittling the struggling student. They look out for one another and ensure everyone has someone to play with or talk to.” “They are cheering each other’s success, are more supportive of each other.” “More of a sense of community [this year]. I see children helping each other and looking after each other to a greater degree from in the classroom to out on the playground at recess [than in past years].” See more comments on this topic in S5 File.

On the other hand, control teachers commented, for example: “[We] have a few children who have a very difficult time acting kind most of the time. This makes it difficult to have a totally close knit community, as these children, while they have progressed, still need significant support to make choices that benefit everyone and not just themselves.” “The students are learning to read and write, but their ability to be well-adjusted and considerate human beings lags behind.” More comments by control teachers are provided in S5 File.

Only 22% of Tools teachers reported the presence of cliques in their classes compared with 89% of control teachers. The difference in the incidence of teacher-reported cliques was significant: χ²(1, N = 18) = 11.99, p < 0.001, odds ratio = 15.77). Fully 89% of control teachers reported in May that there was at least one child in their class who tended to be ostracized or left out; only 33% of Tools teachers reported that. Instances of a student being left out or ostracized were noticeably more common in control versus Tools classrooms: (χ2(1, N = 18) = 4.87, p = 0.02, odds ratio = 3.45). Teachers’ comments echo the stark difference evident in Fig 4 (see S5 File).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Teacher reports of peer rejection and presence of cliques about here.

In-groups and out-groups and peer rejection or exclusion were the norm in control classes and rare in Tools classes.

https://doi.org/10.1371/journal.pone.0222447.g004

Attention-regulation and self-control

Ability to get back to work after a break.

Back in the Fall, most teachers in both groups (89% in each) felt their students were not good at getting back to work after a break. Though comparable in the Fall, the groups differed by the Spring. All Tools teachers reported their students were good at getting back to work after recess and weekends; only 56% of control teachers reported that (χ²(1, N = 18) = 5.31, p < 0.02, odds ratio = 5.28; see Fig 5). Many Tools teachers mentioned how different this was from past years, e.g., “In 20 years I have never been able to come back from school holidays so seamlessly.” This difference already emerged by Spring break (χ2(1, N = 18) = 4.92, p = 0.02, odds ratio = 3.50). Eighty-nine percent of Tools teachers agreed strongly that their children had been good at getting back to work after Spring break; no control teacher strongly endorsed that. Indeed, 56% of control teachers disagreed, saying that their children had not been good at getting back to work after Spring break; only one Tools teacher disagreed.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Teachers’ perceptions of students’ ability to get back to work after a break.

In the Fall, few teachers thought their students were good at getting back to work after a break. By the Spring, Tools teachers were almost twice as likely as control teachers to think their children now were good at getting back to work after a break.

https://doi.org/10.1371/journal.pone.0222447.g005

Ability to work independently, without supervision.

In the Fall, 55% of teachers in both groups said that children in their class were not capable of working on their own at all without supervision, even for a minute. The percentage of teachers endorsing that their children could work on their own for just 1–2 minutes without supervision was 44% for Tools and 33% for control teachers. Only one teacher said her students could work on their own for 3–5 minutes without supervision; that teacher was in the control group. By May, teachers in Tools said their children could be left to work without supervision for far longer than control teachers: F(1,15) = 11.43, p < 0.005, ηp² = 0.43. See Table 2 and Fig 6.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Teachers’ responses to the question, "If someone comes in your room now [in May], how long do you feel you could talk with that person and let the children in your class work on their own without supervision?".

https://doi.org/10.1371/journal.pone.0222447.t002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Ability of children to work on their own, unsupervised.

By May, teachers in Tools felt their children could be left to work without supervision for far longer than did control teachers, although teachers’ estimates of this had been comparable in the Fall.

https://doi.org/10.1371/journal.pone.0222447.g006

In their comments, teachers elaborated at length about this, and how different the experience this year in Tools classes was from previous years. See comments about this by Tools and control teachers in S5 File.

Tools teachers further commented on what the children’s better EF abilities to stay on task and control their attention and behavior has meant for what they can do in class: “The ability of my students to regulate their behaviour and to help those who still require some assistance has allowed me to be able to work with small groups as well as individually with specific students who require additional assistance. I have never been able to effectively do this ever with kindergarten students before.” “They are very self-regulated so I am able to work with a small group without being distracted. This is a wonderful gift.” “I have the freedom to work with small groups and help children learn at their own level; it helps provide students help where they need it and move them further faster. It is definitely more individualized…. Students easily work in small groups and can self-regulate while I work with students who need support.”

Teachers’ feelings about teaching

We asked teachers to rate how they were feeling in May on a scale of 1 (excited about teaching, energized) to 10 (exhausted, burned out, weary) and to rate how they felt looking ahead to the next school year from 1 (excited about starting again, totally enthused) to 10 (not looking forward to it, looking forward to retirement). To both questions, over three-quarters of Tools teachers chose #1 or #2; no control teacher did. They were exhausted. See Fig 7.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Teachers’ feelings about Teaching.

Over three-quarters of Tools teachers, but none of the control teachers, responded in May that they were extremely excited about teaching (choices 1 or 2 on the 10-point scale): χ²(1, N = 18) = 4.99, p = 0.02, odds ratio = 3.58. Similarly, all Tools teachers responded that they were utterly enthused and looking forward to the next school year (choices 1 or 2 on the 10-point scale), while only 2 control teachers selected choice 2: χ²(1, N = 18) = 5.67, p < 0.02, odds ratio = 5.86.

https://doi.org/10.1371/journal.pone.0222447.g007

Comments by Tools teachers indicated that students’ joy in coming to school, excitement about learning, and marked progress were the main contributors to their own excitement about teaching: “I have seen so much success in my students’ learning that I can’t wait to begin teaching again next year now that I have a better understanding of the program and all of its benefits!” “I have enjoyed seeing the students get so excited about coming to school and learning….[M]any students refused to miss school even if they were sick.” “What I have enjoyed most about my class this year is… The smiles and joy.” “What I liked most about teaching this year: Students’ enthusiasm towards learning and their pride in their development.” More comments by teachers are provided in S5 File.

Discussion

This study found that Tools not only improves academic outcomes in reading and writing, but also shows for the first time that Tools also improves EFs in the classroom (being able to stay on task and quickly resume work after a break), markedly reduces teacher burnout and children being ostracized or excluded, and increases the joy students and teachers experience in school.

Limitations of the present study are: (1) Any new program may show benefits simply because it is new. Tools was new here and it was not compared to another new program but to a wait-list control group. (2) Teacher reports should be viewed cautiously because people can see what they hope and expect to see. (3) In the glow of the first year of a program, larger benefits are often seen than in subsequent years. (4) We might have had statistical power to detect group differences in math improvement or differential benefit from Tools in children from lower-income homes had we had more than nine classes per condition. (5) Though we had worked quite hard to match the Tools and non-Tools classes on teacher and student variables, hours of professional development, funds for materials, etc., one difference crept in unbeknownst to us: The Tools teachers arranged to meet together bi-monthly. It is possible that if the teachers in the control group had also met together, their results might have been better and the difference between their results and those from Tools less marked.

Even taking those considerations into account and therefore assuming that gains may appear larger here than they truly are, even if the true gains are half of what was observed, they are still quite impressive whether one looks at objective measures of academic performance, first-person reports of reduced teacher burnout, or teacher reports of student behavior. Results were better for Tools classes across most domains (reading, writing, peer inclusion, children’s ability to get along with, and be kind and helpful to, one another, children’s ability to work independently and stay on task without supervision, their ability to settle down quickly after a break and get back to work, teachers’ renewed joy in teaching, and students’ excitement about learning and joy in coming to school). The one exception was math, where results tended to be better for Tools classes, but not significantly so. The results were better than (a) the same teachers had in previous years and (b) control-group teachers had in the year of the study.

The percentage of children able to write one or more self-composed sentences was almost 300% greater in Tools classes (63% in Tools versus 22% in control classes), though writing skills had been comparable in Tools and control classes in the Fall. Conversely, the percentage of children whose writing skills were no better than the initial sounds of words was almost 300% greater in control classes (8% in Tools versus 23% in control classes). Tools emphasized writing and control classes did not. Thus, it is not surprising that children in Tools showed more advanced writing than control children. The extent to which children in Tools progressed is quite surprising, however—well beyond what anyone in BC public schools had seen previously in kindergarten children and well beyond the upper limits on the BC assessment tool for kindergarten. These results suggest that if writing is encouraged and supported in the way that Tools does, kindergarten children are capable of far more advanced writing than most educators and parents have assumed (and children start kindergarten 6 months younger in BC than in general in the US). (Tools teaches writing before reading, and being able to write is critical for recording what they plan to do in their play scenarios so children in Tools are highly motivated to master that. See S6 File for writing samples).

The developers of Tools had prioritized aligning Tools with Canadian learning standards and styles (this being the first implementation of Tools in any Canadian kindergartens) and had prioritized language skills over math skills for the first year of implementation. Thus, outcomes for language skills were markedly better for Tools versus control children and outcomes for math only marginally better. Importantly, math did not suffer in the Tools classes. There was no tradeoff with language skills being better in Tools classes and math skills worse. Math performance was at least as good, indeed marginally better, among children in Tools versus control classes.

By the end of kindergarten, Tools teachers estimated that their children could continue to work unsupervised for two and a half times longer than control teachers estimated for their students (12.3 versus 5.1 minutes). After breaks, 100% of children in Tools, but only about 50% of control children, could get back to work right away, according to teacher reports. This was echoed in marked differences between Tools and control classes in teachers’ comments about children’s self-regulation, ability to pay attention, and ability to work independently. This speaks to one of the greatest challenges voiced by Grade 1 teachers and one of their most common complaints—children’s poor self-regulation and ability to pay attention. Indeed, teachers report that the task of managing the classroom can lead to high levels of stress and burnout [85]. Children’s ability to pay attention when they enter Grade 1 predicts their later achievement in both math and reading [86, 87]. Children’s ability to work independently is critical to teachers’ ability to give individual attention to a student and for students to be able to work at different levels or follow their unique interests. Indeed, teachers mentioned that before Tools they had had difficulty supporting the more advanced students to move ahead of the rest.

In all control classes but one, teachers reported at least one child was likely to be left out by other children and in-group cliques had formed. In contrast, in all but two or three Tools classes, that was completely absent. Many control-group teachers mentioned in May that were still problems with some children hitting others or refusing to share, but that was no longer present in Tools classes. These findings have implications, we think, for reducing the incidence of bullying and of mental health issues in primary school.

Those large differences are particularly noteworthy because BC emphasizes educating children to be socially responsible citizens who are kind and compassionate. All teachers had this as a goal, but Tools enabled teachers to experience more success in realizing that goal, even though most teachers had received training in the Second Step social-emotional learning program and about half had received training in the MindUp mindfulness and social responsibility program. One would expect differences to be even greater between these Tools classes and classes where prosocial behavior was not a curricular priority.

Control group teachers were wait-list controls. They were looking forward to also being trained on Tools. In the meantime, during the study year, they were given the opportunity to get professional development workshops for free on whatever they wanted. They were thrilled with the three workshops they chose and much appreciated the funds we provided for new materials. There was no indication they were demoralized at not being chosen to be trained on Tools at the outset of this study.

By May, however, all teachers in the control group were indeed exhausted. None felt excited or energized about teaching or excited and enthused in looking ahead to teaching next year. In contrast, almost 80% of Tools teachers did (78% versus 0%). In part that was because the Tools teachers perceived their students as experiencing so much joy in school, progressing so far, and gaining so much confidence and sense of self-efficacy. Indeed, Tools teachers reported seeing improvements in all three core needs identified in self-determination theory [88] (increased feelings of social relatedness [community], autonomy, and perceived competence).

Economically-disadvantaged children benefited (as past studies had demonstrated, e.g. [68, 69]), but for the first time Tools was also tried in schools serving primarily middle and upper-middle income families. Children across the board benefitted from Tools–whether higher or lower socio-economic status (SES) and whether more advanced in academic skills or self-regulation at school entry or not. Outcomes did not differ significantly by teacher characteristics or children’s free-lunch or ESL status.

Children in Tools with weaker reading skills at school entry made greater progress in reading than other children in Tools. (Also, while the results for reading had differed across Tools classes with more versus fewer lower-income children in the Fall (F[1,6] = 5.74, p = 0.03, ηp² = 0.24), those differences largely disappeared by the Spring (F[1,6] = 1.08, NS). Tools, thus, tended to reduce initial disparities separating children, schools, and teachers. Regardless of the SES levels of students in the class, the prevalence of English-language learners, or the experience level of the teacher, by May over half the children in Tools were able to read and write independently. Principals and resource teachers were surprised, when they walked into Tools classrooms in the Spring, to be unable to identify the special needs students.

Our assessment measures did not show greater writing gains for Tools children from lower-SES homes than from more economically-advantaged homes (χ²[1, N = 9] = 3.37, NS). Yet, when the 2 Tools expert trainers from Colorado came for their Spring workshop and were given de-identified samples of the children’s writing, to their surprise they were no better than chance at identifying those from low-income classes and those from middle-income ones, whereas in the Fall the differences had been stark. This suggests that while our measures did not indicate differential progress, the gap appears to have closed at least to some extent. Differences by SES were still present to be sure, but they were noticeably reduced (so much so that no significant differences remained).

Null results for Tools versus comparison conditions were reported at a Society for Research on Educational Effectiveness (SREE) conference [89, 90] although these have never been published in a peer-reviewed journal. At the SREE meeting, Lonigan [89] reported no differential benefit to EFs comparing preschool Tools to his program (Literary Express^™). EFs were not assessed in that study, however, so the report of no difference in EFs outcomes is curious. Many of the schools in that study requested that their district adopt Tools; no school requested that for Literacy Express.

Wilson and Farran [90] reported null results from Year 1 of their study of pre-kindergarten Tools in Tennessee and North Carolina. Theirs was a textbook-perfect research design, but some of their outcome measures were prone to ceiling and floor effects (e.g., most 5-year-olds pass the Dimensional Change Card Sort test). One school district in the Wilson-Farran study was so impressed by the markedly better writing of Tools children that the district used its own funding to have all its teachers trained in Tools. (Assessment of writing had not been part of the research study). Other school districts that had been in the study did likewise because principals and kindergarten teachers felt they observed better social skills and readiness for learning in children who had attended Tools pre-kindergartens versus children from other pre-kindergartens. (The research study had not evaluated children in kindergarten, but only at the beginning and end of pre-kindergarten).

A limitation of the present study was the lack of funding to follow the children into Grade 1 and beyond, and also to include additional cohorts in teachers’ second and third years of implementing Tools, as had been our plan. We had hoped to investigate whether similar results would be replicated with other cohorts and to investigate how long gains would last and whether they might even increase. There is much evidence of academic gains increasing over time from Tools and from other beneficial programs (e.g. [68, 91–94]).

One reason particularly large effects may have been found in the present study is that all teachers in both the Tools and control group had indicated a willingness to learn and implement Tools. Other studies may have assigned teachers to Tools who might have been disinclined to implement it, weakening their effects. Usually teacher preferences are ignored in implementation studies. But teacher preferences can exert large effects! A teacher who is opposed to a program is less likely to do a good job implementing it [95–97]. After a study documents benefits from that program, that same teacher might then willingly implement it.

There are limitations on the possible applicability of the results found here to other contexts: (1) Important differences between early education in BC and the US led the developers of Tools to feel that the implementation of Tools in BC was more developmentally appropriate and a truer implementation of the Tools philosophy than Tools in the US. Without having to worry about high-stakes standardized tests at the end of year, stress levels were lower and Tools could be implemented the way it was intended–following each child’s lead. A particularly important difference to the developers of Tools was the stronger emphasis on play in BC Tools classrooms than in US Tools classrooms. In BC, children had an hour of play daily where they dramatized what they had been reading. They became deeply immersed in it, becoming the characters, and wrote about what they had learned about the lives of the people. Almost all children attained the level of intentional, mature make-believe play that Vygotskians associate with the development of self-regulation. In the US, because of the press for academics, children have only 20–30 minutes, and less as the year progresses, to dramatize stories and they do so only once a week instead of daily.

Clearly a full hour of dramatic make-believe play daily plus time each day for other types of play is not inconsistent with children doing extremely well in kindergarten, since the children in the present study did extremely well. Indeed, it is possible that copious playtime in kindergarten may be critical for laying the groundwork for academic success. This is especially noteworthy since there is enormous pressure on teachers to allow less and less time for play and devote more and more time to direct academic instruction, even in kindergarten [98, 99].

(2) Another limitation on possible generalizability is that most teachers in the present study were experienced. Tools is a demanding curriculum. Teachers in the present study bemoaned the amount of information to learn, e.g.: “The vast amount of materials that accompany the program is a challenge.” “The most challenging thing was implementing everything in the program. Adding a few new things was okay but having to learn everything and teach all new things at once was very challenging.” Tools may work best with teachers with at least a bachelor’s education, as most teachers here had.

This study does not enable one to determine “the active ingredient” of Tools nor which benefits of Tools contribute to making other benefits possible. Our hypothesis is that Tools works because of the gestalt that is Tools and that searching for the key element would be futile and fruitless. We hypothesize that Tools improves EFs because it directly trains, scaffolds, and challenges them, providing numerous opportunities to practice exercising them at progressively more advanced levels, and because it supports them by improving emotional and social well-being. We also hypothesize that Tools improves academic skills by directly targeting them in ingenious ways and because Tools improves EFs and emotional and social well-being.

Most elements of Tools probably affect more than one outcome. For example, scaffolding EF skills not only helps children practice those skills at a more advanced level than they would otherwise be able, thus aiding the development of those skills, but also reduces stress in the classroom. Teachers are less worried about the children not being able to exercise self-control or attention-regulation and children are less worried about being scolded for not exercising those EF skills [76, 100]. Conversely, stress impairs self-control [101, 102] and attention-regulation [50, 103], and reducing stress aids them. Also, by scaffolds increasing the likelihood of success and reducing the incidence of failure, they help build children’s self-confidence and belief in their ability to succeed [76].

Once children have a modicum of self-control and attention-regulation, that makes possible being able to work alone or with another child without the teacher needing to control the class from the front of the room. That makes possible a host of beneficial educational practices such as individualized pacing, instruction, and assessment because all the children doing the same activity together is no longer required [76, 100].

The paired play (pairing each child with every other at least once every week) not only helps each child get to know all the other children better and learn to get along with and work together with each (helping to build a sense of community), but also aids the development of language and EFs through the regulation of one another by verbal correction and feedback [80]. In Tools, each child in a pair gets to play the role of the “checker” and the one being checked, including younger children serving as the checker for older ones. The hands-on learning by working together with another child aids mastery of the academic material [76]. The improved sense of camaraderie in the classroom, which paired play facilitates, probably also aids EFs.

The clearest findings in the present study are: (a) Tools reinvigorated teachers’ enthusiasm for teaching. Those concerned with teacher burnout should take note. Burnout is leading many teachers to leave the profession [25, 104] and it causes many who stay to have less commitment to their job and less patience with the children [24, 104, 105]. Job burnout also contributes to poor health [106, 107]. Indeed, a one-unit increase in burnout score was found to be associated with greater risk for hospital admission for mental health problems and for cardiovascular problems [107]. (b) Teachers perceived Tools as making a big difference and perceived far better outcomes on an array of dimensions (academics; kindness, cooperation, and helping; joy in learning) than in the past. This seems to be a consistent theme across all studies of Tools, even where null findings have been reported. It is unlikely that that these findings are simply a halo effect for a new curriculum, because when compared head-to-head with another new curriculum, Literacy Express, in the study mentioned by Lonigan (89) many teachers requested that the district adopt Tools but none requested that for Literacy Express. It may be that teachers are seeing what they want or expect to see, but in the case of Diamond et al. [69] the teachers expected the district’s new program to yield better results than Tools and the district administration was very dismayed when it did not, since they had put so much effort into developing their new curriculum and were so proud of it. As researchers we need the humility to accept the possibility that teachers are picking up on things our assessment tools might be missing.

(c) Tools markedly improved reading and writing and these findings provide an existence proof that kindergarten children can write at more advanced levels than most had thought–composing sentences of their own creation with advanced vocabulary (e.g., stalagmites and stalactites). Tools teachers in the study said that their experience this year had changed their expectations of what kindergarten children could accomplish, e.g., “Children in kindergarten are capable of so much more than I imagined.” This occurred despite–or perhaps because of–carving out an hour a day for social dramatic play, encouraging other forms of play, and spending as much time on social-emotional growth as on academic growth. Clearly there is no indication whatsoever that play or social emotional learning interfered in any way with academic progress, and might well have aided it.

The findings of the study have relevance to several issues of keen scientific and societal interest: reducing the epidemics of bullying and teacher burnout, increasing student engagement in school, improving academic outcomes, and reducing socioeconomic inequalities in academic performance and EFs.

Supporting information

Acknowledgments

We thank all the teachers who took part in this project and the Vancouver and Surrey School Districts for their willingness and openness to participate.