Evaluation of Evidence-Based Practices in Online Learning

I came across this study courtesy of Donald Clark who reviewed it back in early September. It reports on a meta-analysis of more than 1000 studies of online learning conducted between 1996 and 2008, focusing in on those which contrasted online learning with face-to-face instruction. It was published by the US Department of Education in May. The report can be downloaded here.

I’m going to share with you what I believe were the most important findings:

"Students who took all or part of their class online performed better, on average, than those taking the same course through traditional face-to-face instruction. Interpretations of this result, however, should take into consideration the fact that online and face-to-face conditions generally differed on multiple dimensions, including the amount of time that learners spent on task."

This is a significant result but, as the authors point out, we are not comparing apples with apples here. The main problem is that the effect of using different methods and media can easily be confused. If you want to compare the effectiveness of face-to-face instruction and online instruction as alternative media then you need to hold the method constant. So, if the classroom experience is synchronous and collaborative, then the online experience needs to be the same, which probably limits you to virtual classroom sessions. If past media comparison studies are anything to go by, then the conclusion anyway is likely to be ‘no significant difference’, because media have only a marginal impact on effectiveness – it’s the choice of methods that really makes the difference. If, on the other hand, you want to contrast face-to-face and online communication in terms of its efficiency then there’s no contest – it is, of course, massively cheaper if you are able to avoid the time and trouble involved in moving people to a central training location.

Now researchers probably don’t want to be limited to comparing synchronous collaborative experiences in real and virtual classrooms, because most online learning is actually asynchronous, i.e. self-paced. As a classroom intervention is always synchronous, the issue here is not whether you are moving the learning online, but the change of method, from live to self-paced. Now we know that self-paced learning is typically more flexible, less stressful for the learner, and generally a whole lot quicker, but we also know that self-paced learning won’t work in every instance, certainly not on a stand-alone basis – many situations require interaction with a tutor and/or with fellow learners, and some learning activities really do have to be synchronous.

"The meta-analysis found that, on average, students in online learning conditions performed better than those receiving face-to-face instruction. The difference between student outcomes for online and face-to-face classes — measured as the difference between treatment and control means, divided by the pooled standard deviation — was larger in those studies contrasting conditions that blended elements of online and face-to-face instruction with conditions taught entirely face-to-face. Analysts noted that these blended conditions often included additional learning time and instructional elements not received by students in control conditions. This finding suggests that the positive effects associated with blended learning should not be attributed to the media, per se."

Which makes the point. A blended solution is not more effective because it mixes face-to-face and online media (although this may well make the intervention cheaper), but because it ensures that the right educational and training methods – self-paced, collaborative or instructor-led – can be used for each element in the intervention.

Link to original post


Leave a Reply