Tag: Technology

A web tool to interpret the impact of programs on student social and behavioral outcomes

By Marta Pellegrini, University of Cagliari, Italy

A recent study described the use of an online application to show the results of a meta-analysis of psychological and educational programs in K–12 on student social and behavioral outcomes. The researchers highlighted the huge variability of the effects based on filtering the types of programs, contexts, and students, and proposed a way to better interpret the meta-analysis results and make use of them in future research.

Data come from a meta-analysis of 323 experiments conducted in the U.S. in K–12 on student antisocial behaviors, self-regulation, social and interpersonal adjustment, and personal adjustment. Several characteristics were extracted to describe the study setting, such as the region of study, the type of program, its duration, grade level, ethnicity, and socio-economic status–all characteristics that may play a role in explaining the variability of the program effect.

ES Contextualizer (https://ebcontextualizer.shinyapps.io/EmpBench/) is a web tool that makes it possible for users to select characteristics of a context to produce targeted estimates of the impact of programs on student outcomes. ES Contextualizer can be used by researchers to plan a new study or to facilitate effect size interpretation.The authors described several examples of how to use the web tool. In one of them, a researcher conducted an experiment on the effects of an intervention on middle school students’ social problem-solving skills based on teacher reports and obtained an average effect size of +0.15. Using ES Contextualizer, this estimate can be compared to similar interventions conducted in similar contexts and grade levels. If the effect size is compared to the correct benchmark of +0.05 for middle school students from the ES Contextualizer tool, the researchers can conclude that what they found is a meaningful impact. If the grade level is not selected, the tool shows a benchmark of +0.26 instead, possibly leading the researcher to draw a different conclusion.

The authors recommended using the web tool’s filters to generate more nuanced effect size interpretations and more accurate estimates for use in power analysis.

Using district messaging platforms to reduce chronic absenteeism

By Nathan Storey, Center for Research and Reform in Education, Johns Hopkins University

In the years since COVID, chronic absenteeism has remained a serious concern across the United States, negatively impacting student learning and achievement. Given the breadth of absenteeism, it is beneficial to identify cost-effective means of improving attendance for a large number of students. Musaddiq and colleagues sought to address this through personalized monthly email and text messages sent to parents through the messaging platforms used by four public school districts in the metro-Atlanta area of Georgia. Messages were designed to inform parents about the number of absences their children had and how that number compared to their peers’.

Results showed that for students whose caregivers were sent the messages (n=>28,300), the probability of chronic absenteeism was reduced by 2 percentage points, although not all caregivers received the message; students whose caregivers received the text messages saw greater reductions in end-of-year absences and the probability of chronic absenteeism (a decrease of 4 percentage points) than non-experimental district students (n=>347,300).

The study also pointed to useful practical findings related to reaching parents. The researchers noted that many parents were not reachable through existing district messaging platforms, that emails were more often received than text messages, and that the parents of students who were most in need of improving attendance were the most difficult to reach. These findings highlight that “light-touch, low-cost, and scalable” parental outreach efforts through existing platforms can have a positive impact on chronic absenteeism, but also point to ways to further enhance programmatic impact, for instance working with districts and schools to identify additional correct means of contacting parents, for instance.

Broadband access and educational inequality: Lessons from Chicago Connected

By Cynthia Lake, Center for Research and Reform in Education, Johns Hopkins University

In an EdWorkingPaper released by the Annenberg Institute at Brown University, researchers examined the impact of broadband expansion on student engagement and achievement during the COVID-19 pandemic. The study analyzed the Chicago Connected initiative, which provided free high-speed internet access to thousands of K-12 students in Chicago Public Schools. Using a quasi-experimental design, the researchers investigated whether improved internet access helped bridge educational disparities or exacerbated existing inequalities. They found evidence that high-achieving students benefited academically from broadband access, while lower-achieving students experienced declines in engagement and performance.

The study revealed that students with high pre-pandemic GPAs experienced increased remote learning engagement and higher academic performance after gaining broadband access. However, students with lower pre-pandemic achievement exhibited decreased engagement and lower GPAs, suggesting that access to technology alone did not improve learning outcomes for all students. The researchers attribute this divergence to how students used technology. High-achieving students were more likely to use digital tools for academic purposes, while lower-achieving students faced greater distractions from non-educational content such as social media and gaming. The findings remained consistent even after schools resumed in-person instruction, indicating that broadband access had lasting effects on student performance. These results highlight the complex relationship between technology access and educational equity. While initiatives like Chicago Connected are necessary to reduce digital divides, the study emphasizes that broadband expansion must be paired with targeted supports to ensure that lower-achieving students can benefit. The researchers recommend integrating structured digital literacy programs, academic coaching, and parental guidance to help students navigate online learning effectively. Without such interventions, increased access to technology could unintentionally widen educational disparities rather than closing them.

Effects of educational technology on the performance of disadvantaged students

Marta Pellegrini, University of Cagliari, Italy

A recent meta-analysis examined the effects of technology-based educational interventions on the academic performance of disadvantaged K–12 students. The review included studies conducted in less developed countries or with at least 50% of students from disadvantaged backgrounds, defined by socio-economic status, minority status, or areas of residence. Studies needed to use technology in class, while courses delivered online were excluded.

The review included 72 studies with 740 effect sizes conducted in 24 different countries, although most of the studies took place in the US and in India. Ed-tech interventions showed an average effect on academic performance of +0.20 across all included studies with average effect sizes of +0.23 in more developed countries and +0.18 in less developed countries. Regarding factors in the interventions and school contexts that made them more or less effective, slightly larger effects were found in math and science outcomes compared to humanities. The authors found that Computer-Assisted Learning (CAL; ES = +0.12) and behavioral interventions (ES = +0.09) had larger effects compared to interventions that just provided the access to technology (e.g., internet, tablets). Behavioral approaches that involved sending messages to parents to keep them informed and engaged were particularly effective, especially in communities where education is undervalued and families have limited resources. This highlights the importance of family and community support in helping disadvantaged students succeed academically.