Tag: Randomized evaluation

Assumed competence while overlooking unseen information in decision-making

Feifei Wang, The Centre for Information Technology in Education, The University of Hong Kong

People interpret the same information in various ways, often believing their own views are objectively true while dismissing opposing perspectives as ignorant or biased. This can have serious implications for decisionmakers in education and other fields. A recent study examined how people assume they have enough information to make decisions, even when they are missing key details. For example, a driver may impatiently honk at a car waiting at a stop sign, only to discover an unseen mother pushing her stroller across the street. This reveals how mistaken assumptions about having complete information can lead to misunderstandings.

To investigate the “unknown unknowns” in decision-making, (i.e., we don’t know what we don’t know), Gehlbach and colleagues conducted a study of decision-making. Participants (n=1261) read an article about a school threatened by a drying aquifer, faced with the decision to either merge with another school or remain independent. They were randomly assigned to three groups based on the information presented in the article: pro-merge, pro-separate, and a control that received balanced arguments for both options. Subjects were mostly male (59%) and White (71%), with a mean age of 39.8 years and a median education level of three years of college.

Results showed that compared to the control group, the two treatment groups assumed that they had adequate information and were capable of making informed decisions. Their decisions were heavily influenced by the subset of information they received. They also believed that most other people would arrive at similar decisions. This study suggests that in a world of unreliable information, recognizing what information may be missing and approaching decisions with humility and curiosity can help us better understand others’ perspectives before passing judgment.

School gardens and nutrition education in limited-income Alabama

By Nathan Storey, Center for Research and Reform in Education, Johns Hopkins University

As funding cuts loom over the education system, from the federal level down to local classrooms, it is useful to remember the impact education and schools can have on all aspects of students’ lives. In addition to math and reading, schools have the capacity to change students’ health behaviors and their nutrition practices. A recent study in Alabama, conducted by Sofia Sanchez and colleagues, examined the impact of school gardens and nutrition education on students’ fruit and vegetable consumption. Other studies have highlighted the positive benefits that healthy diets have on academic achievement and social and emotional well-being. However, as Sanchez and her team noted, disparities in diet and nutrition knowledge persist along race and economic lines in Alabama and beyond.

In their study, the researchers conducted a randomized trial of over 4,000 third-grade students from 99 Alabama schools, some with on-site school gardens and others without. After surveying students on their nutrition, treatment students participated in the Body Quest school-based health promotion curriculum, which focused on healthy foods, beverages, and physical activity. Researchers were particularly interested in whether the presence of gardens and BQ provided an additive impact.

Before the intervention, students at schools with gardens already reported higher fruit and vegetable consumption. Using multi-level modeling, the researchers found that BQ nutrition education increased third grade students’ fruit and vegetable intake. Surprisingly, students who participated in BQ but did not have access to a school garden demonstrated the greatest vegetable consumption increases. While not feasible for all schools to maintain gardens, the study suggest that both school and community gardens, along with nutrition education alone, can positively influence student dietary practices.

The effects of CGI professional development on student achievement in grades 3-5

By Susan Davis, Johns Hopkins University

A cluster-randomized trial examined the Year 1 effects of a three-year professional development program for math teachers, the Cognitively Guided Instruction (CGI) 3-5 Program, on third through fifth graders’ achievement in fractions learning.

The study involved 149 teachers in grades 3-5 from 31 schools in 9 districts in Florida who were assigned randomly to the CGI condition or to a wait-list control group. Teachers in the CGI group participated in workshops on number operations, and algebraic thinking, with a focus on fractions. These workshops took place over 5 days in the summer, 2 in the fall, and 2 in the winter. The training focused on math problem types and student strategies, and included teacher-student interactions where teachers applied what they learned, with later reflection and sharing with colleagues. Teachers in the control group continued with business-as-usual instruction.

Students were pre- and posttested using the Elementary Mathematics Student Assessment. Results showed that students of teachers in the CGI program outperformed the control group at post-test (ES=+0.19), which authors cited as a medium effect size equivalent to 1/5 of a school year in grade 3, and 1/3 of a school year in grades 4 and 5. A cost analysis, using Kraft’s schema to relate effect size to program cost, yielded a low per-student cost of $137.25. While CGI 3-5 is designed to be a 3-year program, this study provides important evidence of its one-year effects.

The impact of vision on mental well-being among rural Chinese students

By Winnie Tam, Centre for University and School Partnership, The Chinese University of Hong Kong

Research findings show that visual impairment is linked to mental health issues like depression and anxiety among children, especially when refractive errors go uncorrected. A recent study published in Scientific Reports by Pang and colleagues explored the relationship between visual impairment, academic performance, and mental health among junior high school students in rural China.

The study involved a random sample of 19,425 seventh and eighth grade students from 124 schools in rural China. Researchers collected data on students’ visual acuity, eyeglasses use, academic performance (measured by a standardized math test), and mental health (assessed using the SDQ) in late 2019.

Key findings showed that 37% of the students had visual impairment, 57% of whom did not wear glasses. About 9% of all the students were at high risk for mental health problems based on SDQ scores. Students with visual impairment who did not wear glasses were more likely to experience mental health issues than those who did, and those who wore glasses had fewer mental health concerns than students with normal vision. In addition, wearing eyeglasses boosted students’ aspirations for higher education, regardless of academic performance.

The study highlights the importance of providing eyeglasses to visually-impaired students to support their mental well-being. It suggests that interventions and policies should focus on supplying eyeglasses to enhance students’ overall well-being in low-income rural settings. More on vision and school-based vision remediation may be found in earlier issues of BEIB.