Scientific Research

–The whole story.

An experimental study on The Expert Mathematician is reviewed on the US Department of Education’s “What Works Clearinghouse” website (click for report).

The study, conducted over the majority of a school year, compared a computer-mediated “generative learning” intervention–The Expert Mathematician (TEM)–vs. the UCSMP Transition Mathematics textbook-based program. The two treatment groups of randomly assigned students were taught by the same teacher, using a block schedule model.

The TEM group posted raw gains of 29% on mathematics concepts over the school year.

While students were not officially receiving special ed. support, the teacher indicated that most were “at-risk” for academic failure, though technically, they were “mainstreamed.”

Achievement variables. Students’ acquisition of conceptual knowledge as measured on a standardized mathematics achievement test was the main research interest. Students in both the experimental treatment and control groups achieved statistically significant gains on conceptual measures.. There was no statistically significant difference between posttest scores for the two comparison groups, though raw gains showed an 18% increase for the textbook group, and 29% for the TEM group–the differences owing to much lower pretest scores for the TEM group. Females were overrepresented in the TEM treatment, and accounted for both lower pretest scores and higher total gains than the males in their group. The pretest differences raise a validity issue that is addressed in the final analysis by redistributing pretest scores to equalize pretest totals in both groups–here, eliminating gain differences.

The finding that students in the TEM group came from behind to equalize total achievement with the well-regarded Transition Mathematics control treatment–while experimentally inadmissable–should not be disregarded. In fact, the novel intervention group did accomplish more than the traditional group in the same time period with the same teacher who was widely experienced in and enjoyed teaching the Transition Mathematics program.

From a scientific point of view, even after covarying out pretest differences, TEM passed a crticial test of scientific merit: “First do no harm.” Although the teacher had no previous teaching experience with the TEM program, the program’s “producitivity” equaled that of a highly regarded program in which the teacher was trained and experienced.

On a 7 dimension attitude survey [TEM] is viewed by students as significantly more favorable (than the textbook alternative).

The Department of Education review also doesn’t report affective data, which was considered by the researcher an important part of this study.

A 7 dimension attitude survey probed students’ changes on a variety of psychological and social measures relating to the study of mathematics at the then present time and into the future. While further development of a sound interpretive framework for these affective measures would be useful, the finding that the TEM media—which afforded students greater personal control over the hard work of learning—is viewed by students as significantly more favorable than those that offers less control, has important implications for study.

Total Scores on Mathematical Concepts and Gain Scores on Attitudes About Studying Mathematics Concepts.