What’s the Difference: a Review of Contemporary Research on the Effectiveness of Distance Learning in Higher Education

Journal of Distance Education/ Revue de l'enseignement a distance Spring/Printemps 1999 Vol. 14, No. 1, 102-114 What’s the Difference: A Review of Contemporary Research on the Effectiveness of Distance Learning in Higher Education Ronald Phipps and Jamie Merisotis American Federation of Teachers, National Education Association, 1999, 48 pages Available online at http://www. ihep. com/difference. pdf Mark Bullen With the increasing popularity of web-based instruction, it seems there has been an explosion of published research, evaluation, and other literature related to distance education.For some time now I have found it increasingly difficult to stay on top of this growing body of literature. So when this report first came across my desk (or screen), I felt a sense of relief that finally somebody had taken the time to synthesize and evaluate some of this growing body of distance education research. The stated purpose of this 48-page report is to examine the research on the ef fectiveness of distance education in order to inform public policy. The specific questions the report sought to answer were: 1. What are the findings of the research on the effectiveness of distance education? 2. Are they valid? . Are there gaps in the research that require further investigation? 4. What does the literature suggest for the future? The report is divided into four main sections: 1. What Does the Original Research Say About the Effectiveness of Distance Learning? 2. What Are the Key Shortcomings of the Research? 3. What Are the Gaps in the Research that Require Further Investigation and Information? 4. Implications. It is important to note that the review is limited to material published in the 1990s that dealt with two-way interactive video, one-way prerecorded video, two-way audio/oneway video, and computer-mediated learning.No definitions of these technologies are provided, which is particularly problematic for computer-mediated learning because it could refer to on line technologies as well as stand-alone computer-based technologies, multimedia technologies, text-based computer conferencing, and both synchronous and asynchronous technologies. Not surprisingly, the chapter that reviews the original research concludes that the experimental studies tend to show that distance students perform as well as or better than campus-based students and that the descriptive and ase studies show generally positive student and faculty attitudes. The authors say they analyzed 40 of the â€Å"most important and salient† research studies of the 1990s and that they collected â€Å"several hundred articles, essays, and other writings published in major journals on distance learning. † However, only five journals appear to have been consulted. The list includes one journal that ceased publication in 1993 (Research in Distance Education) and does not include one of the most highly regarded journals in the field, Distance Education. In the review chapte r, only 10 studies are cited.Three of these are from the American Journal of Distance Education, one is from the Journal of Distance Education, four are papers from a regional American distance education conferences, one is a paper from the national American conference of the Association for Educational Communications and Technology, and one is a case study prepared for the California State University system. The limited research on which this review appears to be based is critically important to bear in mind when reading the chapter on the key shortcomings of the research.This chapter concludes that much of the research is flawed and therefore of questionable value. Specifically the report suggests: 1. Extraneous variables were often not controlled for; 2. Researchers failed to use random assignment of subjects to treatment and control groups; 3. The validity and reliability of measurement instruments are suspect; 4. Reactive effects of subjects were not properly controlled for. Tw o studies not cited in the review chapter are cited here, which brings the total number of studies cited to 12.One of the additional studies is from a paper presented at a regional American distance education conference, the other from a meeting of the National Association for Research in Science Teaching. What stands out clearly in this chapter is the objectivist/quantitative frame of reference from which the research is critiqued. There is no allowance for research informed by subjectivist epistemologies that use interpretivist/qualitative methods. In fact, earlier in the report only four research approaches are described: descriptive, case study, correlational, and experimental.I suspect the criticisms of the research in the report are not far off the mark. Several years ago I conducted a similar review and reached similar conclusions (Bullen, 1990). However, the answer does not lie in tightening the methodological screws, but in rethinking the whole research approach. Ethical an d practical considerations make it almost impossible to conduct truly experimental studies in education. Students cannot be randomly assigned to control and treatment groups in these kinds of situations.Controlling extraneous variables means that technologies cannot be used in ways that take advantage of their unique characteristics. For example, imposing this kind of control when comparing video with classroom instruction would mean simply producing a videorecording of the classroom presentation for the distance students instead of exploiting the unique symbol system offered by video. Another implicit assumption underlying this report is that classroom-based instruction is the norm and that any new technology must measure up to this.The report suggests that the research on the effectiveness of distance education places too much emphasis â€Å"on the utopian possibilities of the technology and its potential to do as well as classroombased instruction. But not enough pragmatism has been applied to allow for a discussion of distance learning’s practical implications as a supplement to enhance teaching and learning† (p. 30). In fact there is probably far less research on the effectiveness of traditional classroom-based instruction at the higher education level than there is research that focuses on distance education.The two chapters that deal with the gaps in the literature and the implications are probably the most useful, even if they are based on a rather weak foundation. Among the more useful recommendations for further research are the following: 1. More emphasis should be placed on the evaluation of whole programs rather than single courses; 2. There needs to be more emphasis on individual differences such as gender, age, educational experience, motivation and learning style; 3. Research should focus on the interaction of multiple technologies rather than the impact of single technologies.The report concludes with three broad implications: 1 Access is more than a technical issue: it depends on quality and student skills to use the technologies; 2 Faculty roles will change from content expert to a combination of content expert, learning process design expert, and process implementation manager; and 3 Technology is often not nearly as important a factor as issues such as learning tasks, learner characteristics, student motivation, and the instructor. Reports that thoroughly review and analyze different strands of the literature help usy distance education practitioners immensely. Although this report makes some valuable recommendations for future research, its narrow scope, limited reach, and epistemological bias mean it does little to help us stay on top of the literature. Reference Bullen, M. (1990). Learner responses to television in distance education: The need for a qualitative approach to research. In B. Clough (Ed. ), Proceedings of the ninth annual conference of the Canadian Association for the Study of Adult Educ ation (pp. 48-53). Victoria, BC: University of Victoria.Mark Bullen Mark Bullen is Assistant Director in the Distance Education and Technology division of Continuing Studies at the University of British Columbia. He assists in the planning and management of the unit, participates in the strategic planning for the development of distance education and distributed learning programs and courses, and provides leadership in the application of educational technology to the design and development of distance education and distributed learning courses and other educational materials (email: mark. [email  protected] ca).