Home Online Education Access and Affordability Interactive Learning Online at Public Universities: Evidence from Randomized Trials

Interactive Learning Online at Public Universities: Evidence from Randomized Trials


Researchers at Ithaka S+R including William G. Bowen (former president of Princeton University), Matthew M. Chingos (also a fellow at the Brookings Institution’s Brown Center on Education Policy), Kelly A. Lack, and Thomas I. Nygren, have followed Ithaka S+R’s recent report titled “Barriers to Adoption of Online Learning Systems in U.S. Higher Education” with a second that reports the findings of a series of randomized trials related to online learning conducted at six public universities.  This second report titled, “Interactive Learning Online at Public Universities: Evidence from Randomized Trials,” (and written by the researchers) attempts to address the concern of negatively impacting student learning by utilizing an online educational format.  While the authors caution readers against believing that further research is not needed on the effectiveness of online learning,, they state that their findings “show comparable learning outcomes…with a promise of cost savings and productivity gains over time.” 

In conducting their research, the authors state that they were attempting to address one of the concerns consistently noted in their interviews for the first report, namely that online learning reduces the effectiveness of teaching and learning and jeopardizes student learning outcomes.  At the six institutions at which the research was conducted, researchers arranged for the exact same introductory statistics course to be taught.  At each, a control group was enrolled in a traditional classroom for the course while a “treatment” group was enrolled in a “hybrid course using a prototype machine-guided mode of instruction developed at Carnegie Mellon University.”  The treatment group took an online version of the course but also met face-to-face for one hour each week.  Students were randomized after agreeing to participate in the study and all participating students took the Comprehensive Assessment of Outcomes in a First Statistics Course (CAOS) test of statistical literacy at the beginning of the course.  At the end of the course, students again took the CAOS test and completed a questionnaire regarding their experience in the course. 

To the best of the authors’ knowledge, no other report has approached the question of online learning outcomes in this type of methodological quantitative manner.  This may be due to the difficulties in controlling the many variables that exist in the learning environment.  The researchers were able to control most of the variables involved (noting, for example, that their attempt at randomization was successful in that the control and treatment groups were equally distributed with similar characteristics among students) but, to be clear, there were some variables that the researchers were not able to control.  For example, the researchers were not able to randomize instructors for either group and could not impact the variable of instructor quality.  Despite the uncontrollable variables, the authors feel confident that their findings are reliable and are a testament to the potential of online learning in U.S. higher education.

The findings in the report indicate that online learning provides learning outcomes at least as effective as those from traditional classroom settings.  They state, “We find no statistically significant difference in learning outcomes between students in the traditional and hybrid-format sections.”  The students in the hybrid course actually performed slightly better than the traditional course students did on three of the outcomes to be studied.  They achieved “pass rates that were about three percentage points higher, CAOS scores about one percentage point higher, and final exam scores two percentage points higher.”  The authors admit that none of these results stands the rigorous tests of statistical analysis and are not considered statistically significant.  The authors state, however, that “at the minimum, this study supports a ‘no-harm-done’ conclusion…” 

The implications for this in the world of higher education are significant.  With the advent of technology, productivity in general terms has increased.  While there are significant startup costs associated with implementing an online learning platform, over time the appropriate use of online learning and its tools and techniques can decrease input while keeping outputs at least the same.  In this way, colleges and universities should be able to increase overall productivity.  In a time of dwindling state and federal funding for higher education and nightmarish budget crises at many colleges and universities, increasing productivity is critical.  Considering the upfront costs of implementing an online learning platform, the authors suggest that colleges and universities conduct “cost probes” to determine the actual cost and return on investment.  In a number of cases, online education has been implemented at colleges and universities with the help of grants from private organizations and other groups. 

The authors note that another benefit from online education is a decrease in space required. They state that “the hybrid course requires 67 percent to 75 percent less classroom use than the traditional course…”  The maintenance of buildings – not to mention the cost of electricity and water required for those buildings (the rates of which are rising for both across the country) – is a sizable chunk of any university budget.  When students are moved from classrooms to the virtual world, the utility costs for a building will likely decrease.  Alternatively, since it is not feasible for colleges and universities to demolish or sell their buildings with the advent of online learning, the authors suggest that “…in the long run, using hybrid models for some large introductory courses would allow institutions to expand enrollment without a commensurate increase in space costs – a major cost savings (cost avoidance) relative to what institutions would have had to spend had they stayed with a traditional model of instruction.” 

More effective utilization of space leads to the ability to increase total enrollments while also potentially increasing access to all students, including for minorities as well as non-traditional learners (including adult learners who may be juggling fulltime jobs, families, and other obligations).  Online education allows far greater flexibility in scheduling for students leading to expanded access as well.  Though in the first report from Ithaka S+R the authors note that many faculty view the advent of online education as a diminishing of their roles, the authors of the second report respond to note that online education can free up a professor’s time for research, additional (perhaps advanced level) courses, and more one-on-one time for students needing further help with a course.  In this way, the instructor’s role arguably increases the faculty member’s role in the learning process. 

Online education continues to move through various stages of development at multiple institutions.  There are a variety of types of online learning, from hybrid courses like the one used in the Ithaka S+R study to fully asynchronous online environments like at American Public University System (APUS) where more than 100,000 students study from all 50 states and more than 100 countries and are never required to come to campus for face-to-face instruction.  Increasing numbers of students are enrolling in online courses as evidenced in the recent report published by Babson Survey Research Group (in conjunction with other higher education groups) which found that in Fall 2010, more than 6.1 million students took at least one online class, representing a 10.1 percent increase over online enrollments in Fall 2009.  Previous articles on this blog have documented the economic crisis facing higher education and impacting the affordability of higher education for the majority of the nation’s high school graduates (see “Impact of the Economy on Higher Education”).  Reports like these last two published by Ithaka S+R provide additional legitimacy to online learning as an effective – and more cost effective (at least in the long term) – alternative to traditional forms of higher education and should be embraced by leaders of public institutions faced with capping enrollments, pressure to decrease costs, and decreasing funding from the states.



Wally Boston Dr. Wallace E. Boston was appointed President and Chief Executive Officer of American Public University System (APUS) and its parent company, American Public Education, Inc. (APEI) in July 2004. He joined APUS as its Executive Vice President and Chief Financial Officer in 2002. In September 2019, Dr. Boston retired as CEO of APEI and retired as APUS President in August 2020. Dr. Boston guided APUS through its successful initial accreditation with the Higher Learning Commission of the North Central Association in 2006 and ten-year reaccreditation in 2011. In November 2007, he led APEI to an initial public offering on the NASDAQ Exchange. For four years from 2009 through 2012, APEI was ranked in Forbes' Top 10 list of America's Best Small Public Companies. During his tenure as president, APUS grew to over 85,000 students, 200 degree and certificate programs, and approximately 100,000 alumni. While serving as APEI CEO and APUS President, Dr. Boston was a board member of APEI, APUS, Hondros College of Nursing, and Fidelis, Inc. Dr. Boston continues to serve as a member of the Board of Advisors of the National Institute for Learning Outcomes Assessment (NILOA), a member of the Board of Overseers of the University of Pennsylvania’s Graduate School of Education, and as a member of the board of New Horizons Worldwide. He has authored and co-authored papers on the topic of online post-secondary student retention, and is a frequent speaker on the impact of technology on higher education. Dr. Boston is a past Treasurer of the Board of Trustees of the McDonogh School, a private K-12 school in Baltimore. In his career prior to APEI and APUS, Dr. Boston served as either CFO, COO, or CEO of Meridian Healthcare, Manor Healthcare, Neighborcare Pharmacies, and Sun Healthcare Group. Dr. Boston is a Certified Public Accountant, Certified Management Accountant, and Chartered Global Management Accountant. He earned an A.B. degree in History from Duke University, an MBA in Marketing and Accounting from Tulane University’s Freeman School of Business Administration, and a Doctorate in Higher Education Management from the University of Pennsylvania’s Graduate School of Education. In 2008, the Board of Trustees of APUS awarded him a Doctorate in Business Administration, honoris causa, and, in April 2017, also bestowed him with the title President Emeritus. In August 2020, the Board of Trustees of APUS appointed him Trustee Emeritus. In November 2020, the Board of Trustees announced that the APUS School of Business would be renamed the Dr. Wallace E Boston School of Business in recognition of Dr. Boston's service to the university. Dr. Boston lives with his family in Austin, Texas.


Your email address will not be published. Required fields are marked *