Programa BEM: Brincar Ensina a Mudar (Play Teaches Change) - Center on the Developing Child at Harvard University

Program Description

Programa BEM (Play Teaches Change) is a play-focused, online parenting program designed to build parent capabilities to interact with their children through play in support of healthy child development. Through the program, mothers and other female caregivers watch video classes and receive text and audio messages that demonstrate opportunities to integrate play into their daily routine using resources available in their homes. The program is designed to be a low-cost, scalable resource.

Project Description

A mother and daughter play together as part of Programa BEM.

This project is part of the first cohort of the iLab Primeira Infância, FOI’s innovation cluster in Brazil. To date, the project has conducted two main feasibility tests and one pilot study. The goal of the feasibility tests was to learn how the program would be accepted by participants and to road test the program’s delivery channel and technology platform (WhatsApp). The pilot study implemented and evaluated the program with 12 mothers of young children facing adversity in São Paulo, Brazil with the goal of beginning to investigate the program’s theory of change. Participants received Programa BEM’s content through 17 video classes and 72 text and audio messages.

Project Timeline

Click to view larger.

Evaluation

Programa BEM Caregiver Annual Income — Caregiver Annual Income

Programa BEM Caregiver Skin Color — Caregiver Skin Color

Programa BEM Caregiver Employment — Caregiver Employment

Programa BEM Caregiver Education — Caregiver Education

Data was collected before and after program participation by trained data collectors during home visits using a combination of program-specific questionnaires, standardized scales, and coded observations from video recordings.
Programa BEM’s intended population to serve is female caregivers of children ages 12-23 months living in urban contexts and facing adversity. The majority of the project sample served reported a low household income of less than R$36,000 per year, low educational attainment (no more than a high school diploma), and lack of full-time employment. However, about a third of the sample had an annual income and educational attainment that was higher than expected. The program is also intended to serve primarily participants of color, yet the majority of the project participants reported their skin color as “White.” Overall, the sample served in the project did not completely align with the population Programa BEM intended to serve, as a subset of participants faced less adversity than anticipated. Moving forward, the team will focus efforts to recruit families facing adversity.

Theory of Change

The Programa BEM team found strong preliminary evidence of change for both program targets (knowledge of how to incorporate play into the daily routine and frequency of play during the daily routine) and two of the outcomes (child development and quality of the dyadic interaction). Two outcomes (parental sense of competence and caregiver emotional regulation) showed weak or no evidence of change. The team is considering removing these outcomes from the theory of change because they are beyond the scope of the program’s focus. Preliminary evidence suggests that participants who engaged more with the program (watching 40% or more of the videos) benefited more, indicating that program engagement might be a moderator of program impact.

In the graphic below, hover over each of the targets, outcomes, and moderators to get more in-depth information about Programa BEM’s theory of change.

The team considered multiple sources of information in determining the strength of the evidence of change observed. These included measures of the statistical significance (e.g., p-value) and magnitude (e.g., effect size) of change on average, as well as the pattern of change for individual participants. To maximize learning, quantitative evidence was supplemented by “practice-based evidence,” such as participant testimonials and informal observations, to inform the interpretation of quantitative data patterns. Given the small size of the sample, findings are considered preliminary and should be interpreted with caution.

Program and Materials Development

The development of program materials focused on creating the digital content to be shared with participants, which included video lessons and accompanying tips sent via audio and text messaging through WhatsApp. The content of the videos and messages was tested during the initial feasibility test. Based on participants’ feedback, this approach was determined to be feasible and acceptable to participants. The Programa BEM team also made progress toward creating an implementation guide, which documents how to implement the program effectively.

Key Lessons Learned

The program strategies are feasible and acceptable.

Promising preliminary evidence of impact warrants further investigation.

Program engagement might help explain who benefits most from this program.

Potential Next Steps

Continue to investigate the impact of the program in a study using greater methodological rigor.

Incorporate strategies to encourage program engagement.

Focus efforts to recruit families facing adversity.