When I look at educational research about instructional strategies, I’m concerned with how often the researchers ignore important controls. They confound their data and then draw invalid conclusions.

I just read some research in which students were taught the same math content but using two different approaches:

• Group A was taught “traditionally”, which included teacher-led, direct instruction;
• Group B was taught with a student-directed approach and a specific context.

After read the descriptions of the two groups, what conclusions could you draw if one group outperformed the other group?

Unfortunately…

…the researchers concluded that the context they used was important for student learning. They admitted that the specific context required a very different instructional approach, but they attributed the achievement differences to the “theme” of the task.

Confound it

That’s really not good enough. You can’t have two major differences between groups and then point to either difference as the cause. In fact, you can’t conclude much of anything from data like this.

Fail.

Do it right

Only change one thing at a time. If you need to change more, try a third group (here, you can have a group with a student-directed approach but without the contextual restriction). Then you’ll be able to tell if the difference is due to the approach or the approach with the context.