Dissecting a Glowing Report on Online Learning

Late last week “Inside Digital Learning” summarized a new report from Arizona State University, on a study that examined well-regarded online offerings at six institutions and suggested that those offerings are helping make education more affordable and of higher quality for students, as well as cheaper for institutions.

“Inside Digital Learning” wondered how the report would be received among those who see online education as a massive force for positive change, and by those who view it a bit more skeptically. We posed the following questions to a range of experts:

• Does this report represent another signal that online education can be a viable business model for many institutions and also help students learn better in the process? Are the institutions in the study outliers in a significant way? Are there limitations for online programs that the report hints at or doesn’t address?
• How should institutions react to the findings in this report, and in what ways should they adjust their strategies as a result (if at all)?
• What can institutions or administrators that have struggled with or remain skeptical of online education take away from this report? Do these institutions represent successful exceptions to the rule, or a model for what the rest of the landscape can become?

Check out their responses and then follow up with yours in the comments below.

Deb Adair, executive director, Quality Matters

I enjoyed reading this well-done and thoughtful case study of six successful approaches to digital learning. The case study methodology provides a refreshingly deep view of a set of enterprise-level operations. Some of the conclusions of this work are supported by the data coming out of the Changing Landscape of Online Education (CHLOE) surveys. Both CHLOE 1 and 2 identified many differences between “enterprise-level” institutions, like those in this study, and institutions with online programs operating at a much smaller scale.

If the purpose of the study was to specifically identify the factors leading to success, I would quibble. When you only look at the most successful cases, it’s hard to draw any evidence-based conclusions about what might lead to success for other institutions. This seems to me to be a minor point, because the recommendations drawn by the authors have a great deal of common-sense appeal. In fact, you would hardly need to read through the case studies to be convinced of their applicability. On the other hand, it would be a loss to skip the details about how these organizations go about the business of digital learning. There is much here to inform the decisions and actions of other administrators.

One aspect, of particular significance to me, is the focus of these institutions on “getting it right” and ensuring they are offering high-quality online offerings. In fact, five out of the six institutions are Quality Matters members, and all have developed quality-assurance tools and processes to ensure that their online courses and programs reflect the affordances of digital technology and address its limitations. The CHLOE 2 survey supports the idea that the largest online programs prioritize quality enhancements in their strategic plans.

I would also urge other institutions and administrators to adopt/adapt the framework for [return on investment] provided by the authors. This is a key takeaway that could help all institutions articulate and prioritize their goals and justify the necessary levels of support. The analysis of the economics of digital learning was particularly compelling (in spite of the use of grades as the single measure of student learning). There are variables here that are rarely used in an evaluation of online program costs, but would provide significantly better insights to institutions if they were. The side-by-side comparison of the real costs for students and for institutions provides a nicely balanced view for evaluating the cost of digital learning in comparison to traditional face-to-face. The CHLOE surveys support the finding in this study that online learning is considered to be a revenue generator for the majority of institutions; however, beliefs about the costs of online education justified higher tuition and/or fees for online learners. The study's ROI framework could provide administrators with a clearer perspective on the relative cost of online education, differentials in tuition and fees, and the relative benefits for institutions and students alike.

Jill Buban, senior director of research and innovation, Online Learning Consortium

This study offers a necessary critical lens on digital learning at an array of institutions which, in many ways, lead the field and act as incubators of digital learning innovations. What I especially like about the sample of institutions is that it includes research institutions and community colleges, or, in the case of Rio Salado [Community College], a state system. The inclusion of private and for-profit institutions would have strengthened the study; however, this sample includes institutions who are geographically diverse, have missions focused on serving diverse populations and have a long history of serving learners online. For these reasons, the study provides the ability for other institutions to find ways to try to replicate outcomes from institutions that are like their own in terms of the student population that they serve.

There are so many takeaways in this report! From financial cost savings to student support techniques to digital courseware strategies, administrators, faculty, instructional designers and student support specialists should be reviewing this report to determine ways in which they might strengthen their digital learning programs, in terms of both learning and financial outcomes.

Supporters and skeptics of online learning and/or digital learning strategies should take note of the models illustrated in this study and determine what strategies might work well on their campuses.

Veronica Diaz, director of professional learning, Educause

For some time, the research has shown that both students and faculty members prefer a blended learning approach over purely face-to-face or purely online. Not only do today’s learners prefer a blended learning approach, with at least some online components, but they do better in that modality than in any other. But to experience the results the University of Central Florida and others have, several key components must be present -- components that are well illustrated in the cited case studies: faculty training, elearning strategic priority, mission-critical elearning systems, adaptable elearning, elearning investment and centralized elearning services. Deploying online learning in an effective manner requires significant investment, not only in human resources, but also in tools and systems.

Institutions in the study aren’t necessarily unique, but over the years, they’ve developed best practices in many critical areas, such as policies and governance; ongoing evaluation and training; outcomes assessment; readiness for all involved in supporting online learning; and significant investments in learner support and in support services. It’s not enough for all of these components to be in place. Institutions have to make a concerted effort to work across boundaries and develop new models that have been proven to produce positive outcomes. There has to be synergy across the organization.

Some of the organizations in the study developed and evolved their models over time to produce the results they have today. Other organizations began their programs, from the ground up, with best practices in mind. For instance, some institutions experimented with developing a few faculty-created online courses at a time in select departments, while others launched entire degree programs that were completely online from the start. Some organizations started with online programs but offered online students face-to-face support services, while other organizations made the online learning experience, including support services like registration and tutoring, completely online. Online learning models, even successful ones, need to be adapted to serve students’ needs and local cultures, but there are many things institutions can take from these and other successful cases to save resources and time as they develop or evolve their own programs.

Another key development, although still in an emergent state, that can significantly support online programs is learning analytics. Today’s online programs can experience positive outcomes and quickly adapt their strategy as needed more quickly than before with the use of data, analysis and predictive modeling to improve teaching and learning. Analytics models use data from various sources -- often aggregating data in new ways -- to help students and institutions understand past and present academic performance and predict future performance. These insights form the basis of feedback, recommendations and interventions to improve student outcomes. Learning analytics can also drive discovery in other avenues, including personalized learning, pedagogical practices, curriculum development, institutional planning and research.

Sharon Goldstein, campus operating officer, Berkeley College Online

We have found at Berkeley College that students who take both online and on-site courses show the highest rate of persistence.

Higher education institutions should continue to embrace the online quality model to provide students with the greatest degree of flexibility, especially as more nontraditional students seek to earn college degrees. At the same time, institutions should be careful to follow a quality model for course development and faculty training. The online platform alone does not guarantee effective teaching and learning.

Faculty training in online pedagogy should be a part of the faculty development strategy. The use of online tools and training should be encouraged to ensure that faculty are properly trained in online pedagogy. Berkeley College has an extensive 50-plus-hour Online Faculty Training Program that goes well beyond the basics of the learning management system. The training includes pedagogical techniques to promote student use of discussion boards and engagement overall. The training also requires faculty to take on a student role by submitting assignments, using the plagiarism tools and posting in discussion boards.

Institutions should be mindful to employ a team of instructional designers who work with faculty to design and build courses that meet stated learning objectives. The course objectives and assignments should be the same regardless of whether they are on-site or online.

Online education is serving millions of students successfully each year. As the Inside Higher Ed article states, student outcomes are often the same or better than on-site courses. Learning management systems have a plethora of engagement tools built in, allowing for highly engaged interactions between faculty and student, and student to student. Instructional designers work with the faculty in building courses that meet stated learning objectives. In addition, institutions should include a network of support services designed to enhance and bolster student success.

Berkeley College Online offers the same support for online students as we do for on-site students, including academic advisement, career services, library services, a center for academic success, student development and campus life, and personal counseling. Co-curricular activities play a role in engaging and retaining students. At Berkeley College, new students can benefit from an engaging and interactive online orientation, and all students are able to participate throughout the semester in online programming. An online book club, a Virtual Honor Society induction ceremony and a virtual commencement, as well as various community service opportunities, are among the ways we engage online students.

Institutions that seek to ensure the delivery of high-quality online programs can pursue certification by national organizations including Quality Matters, the Online Learning Consortium and the United States Distance Learning Association.

Shanna Smith Jaggars, director of student success research for the Office of Distance Education and E-Learning, Ohio State University in Columbus

The ASU report is limited in its utility for college leaders, for several reasons. First, it neglects an existing body of evidence showing that the impacts of online learning are very different for different populations. For example, a 30-year-old middle-income working mother will probably perform as well or better online, while a low-income 19-year-old will probably perform more poorly online than face-to-face. Second, the report’s authors explicitly selected six institutions with very large and high-quality online programs, and at those institutions, they observed a number of good outcomes.

Yet it’s entirely possible that many other institutions have experienced similarly good or better outcomes with small or nonexistent online programs, or have invested in large and expensive online programs without good outcomes. Third, the six institutions’ online learning outcomes are reported only in a descriptive way, without controlling for factors such as gender, age or underrepresented minority status, which makes it impossible to draw conclusions about the effectiveness of the colleges’ online courses or programs in comparison to their face-to-face programs. For example, the report skims over the fact that underrepresented minority students were less likely to be online at these institutions, and does not discuss these students’ outcomes in online versus face-to-face courses.

Given these problems, the ASU report doesn’t provide much new actionable evidence for college administrators. However, it does make two excellent points. First, depending on local contextual factors, some colleges can strongly increase enrollment and revenue through careful and strategic expansion of online learning. And second, revenues generated through online learning should be invested into instructional quality, through such mechanisms as team-based course design and in-depth ongoing professional development. In the best-case scenario, these practical and cultural changes to instruction will extend not only to online courses, but also to face-to-face courses.

For example, this year U.S. News & World Report ranked Ohio State University as No. 1 in undergraduate online degree programs, and No. 2 for online graduate nursing programs. With a commitment to quality over quantity, Ohio State implements many of the quality recommendations set forth in the ASU report while increasing its online offerings at a steady but thoughtful pace. Revenues from Ohio State’s online programs support a larger strategy for instructional innovation across the entire university, including platforms, tools and training that impact not only online but also face-to-face instruction. It’s all part of the university’s goal to become as well-known for the quality of its teaching as for the quality of its research.

The institutions in the ASU report have other strategic goals, which led them to develop large online programs. But that doesn’t imply all colleges should develop similarly large programs. Instead, colleges should first consider how digital learning (including technology-enabled active learning in face-to-face classrooms) supports their particular goals related to student access and success. Then they should develop, and carefully and continuously assess, their digital learning program against those goals. Without that lodestar to guide them, colleges can unintentionally undercut access or success for their most vulnerable students.

Clare McCann, deputy director for federal policy, New America higher education initiative

There’s no doubt that distance education is a growing platform for students enrolled in higher education -- and, increasingly, a sizable source of income for colleges. But plenty of questions remain, even after the release of this latest study. Chief among those is whether institutions are willing and able to put students, not their bottom lines, first as they set up and expand their online programs.

Without question, online college courses have the potential to increase access and lower costs for students juggling busy schedules and competing demands, or who don’t have nearby brick-and-mortar alternatives. But that potential has often not been realized. Online programs have shown huge variation in quality, with some showing comparable outcomes to face-to-face programs, and others falling well short of good outcomes for their students. For instance, research has found that students in online courses have worse grades, less learning and lower rates of retention than other students. And some colleges have used their online programs to reduce, or even all but eliminate, the kinds of high-quality interactions with professors and peers that studies suggest are necessary for deeper learning.

And most importantly, these online outcomes are especially poor among -- and create the biggest problems for -- the most vulnerable students, including those who come to college underprepared. Over one in three first-time students today, and half of community college students, take at least one developmental education course while in college. Yet researchers have found that, while those students may succeed in hybrid online and in-person courses, they are at a higher risk for dropping out of online courses than with face-to-face courses.

Moreover, while much has been made of the potential cost savings for online education, far too many institutions either fail to realize those savings or refuse to pass them on to their students. Most colleges surveyed, the WICHE Cooperative for Educational Technologies found, charge as much -- or, amazingly, more -- to their online students than to their brick-and-mortar students. Of course, there may be high start-up costs to setting up an online program. But particularly where programs have increased the number of students assigned to a faculty member to reduce their instructor costs, that’s hard to justify.

Colleges enrolling students in online programs need to be committed to doing more to recognize the significant risk to students, especially low-income and underprepared students, of online education. They must be willing to engage in full transparency about their outcomes for students and strict accountability for poor performance; and colleges must serve their students well, even where it carries added costs. If online education is just a means to an end for colleges -- a steady stream of dollar signs logging on to the website -- it’s bound to exacerbate, rather than shrink, the already substantial equity gaps in higher education.

Johann Neem, professor of history, Western Washington University

The report “Making Digital Learning Work” (2018) concludes that combining face-to-face with online courses improves retention and graduation rates and makes college cheaper and faster. The report’s conclusions require more context. For example, the report argues that students who take some online courses graduate earlier than those who do not, but it’s not clear from the report whether the results are due to student, course or institutional factors. No doubt, online courses provide flexibility for nontraditional students. Yet the report opens with the promise that online courses can be used by institutions for “growing revenue.” Are schools incentivized to move online because it’s profitable? We need to know whether the results reflect something about the courses or the fact that institutions have underinvested in faculty and classrooms to meet students’ needs.

When examining grades, the results varied, but the report posits that students in “mixed-modality,” or hybrid, courses perform best. Hybridization is older than computers; any professor who asks students to read a book outside class uses technology to improve learning. What makes digital tools different? The report rightly notes that they can free up class time for more active teaching. I would also hypothesize that new technologies address the fact that in education, as in health care, providers cannot ensure quality. Car companies can build better cars in-house, but professors cannot unless students do their work. Yet today’s college students spend less time on course work outside class than in the past. For better or worse, with software, professors can hold students accountable.

Additionally, it’s not clear that technology makes college cheaper if, as the report concludes, much of the savings come by relying on contingent faculty to teach larger classes. These savings have nothing to do with technology. Computers have not made tenure nor academic freedom obsolete. Professors online must be as free to choose what and how they teach as on campus. Moreover, scalability is limited if we care about the quality of students’ experiences. Assessing complicated work, such as papers and discussions, requires professors’ expertise, wisdom and judgment. And all students, whether online or on campus, deserve opportunities to interact closely with their teachers.

This report, like so many, opens with reference to the crisis of rising education costs to justify its conclusions. This narrative must be rejected. The biggest driver of rising tuition is not cost but public defunding. Moreover, while tuition is too high, there’s no student loan crisis. A 2014 study concluded that the average student loan was about the price of a new car, and that graduates’ lifetime earnings have grown faster than their debt burden. The most vulnerable students are those who take out loans but, for multiple reasons, do not complete degrees. These students need more support, and digital tools may have a role to play, but technology should not be used to solve a manufactured crisis, nor is it an excuse to redistribute authority from tenure-line faculty to administrators and third-party vendors.

Laurie Quinn, provost and senior vice president for academics, Champlain College

The recent report from Arizona State University and the Boston Consulting Group profiles six institutional success stories in online education. Approaching the question of return on investment with admirable comprehensiveness, the authors wisely avoid the dead-end thinking that casts digital learning as either a quick fix for the frayed cost model of traditional higher education or a necessary compromise in expanding access to college.

“Making Digital Learning Work” spotlights successful online learning endeavors that are showing steady returns for the diverse groups of students who persist and complete their degrees at these institutions and for the institutions themselves. Student outcomes are the touchstone of our work in education, and the report’s meta-analyses contribute significantly to our understanding of how online learning is serving those earning credits and degrees. In one case study, Kentucky Community and Technical Colleges System shows important graduation rate gains -- 25 percent better for students who take a significant share of their courses online as compared to in person. In another, Rio Salado’s strong student success and academic quality commitments coexist with a lean and innovative instructional cost model. The report is refreshing in its eclectic selection, delving into what fueled each institution’s trajectory and recognizing that there’s no formula while culling some sound best-practice recommendations.

My takeaways from this research are informed by our experiences since 1993 at Champlain College Online, where we have more than tripled enrollment online in the past five years. Advice I’d offer late-entrant institutional hopefuls to the landscape of online education offerings, or those still waiting for their online investments to pay dividends:

• It’s late. The institutions in the best position to succeed online have experience and momentum on our side. An institution just now entering the market or developing its focus needs a laser-clear strategy for what’s being offered (hint: not everything!), to whom, how and why.
• Online isn’t the innovation story. Online success is a manifestation of student-focused innovation cultures that tend to learning first, both students’ and our own. We continually reject complacency in favor of curiosity. That’s what the success stories in online education have in common.
• Faculty are central to a thriving portfolio, and there’s more than one good way to configure faculty roles in a successful online infrastructure.
• Quality, quality, quality. While encouraging evidence mounts with each new outcomes study, online remains the Ginger Rogers of higher education: we have to be able to prove quality dancing backward and wearing heels.

This advice is founded on the recognition that there are still a lot of people considering or reconsidering college who want to close the hope gap, a term we’ve coined at our institution for the fusion of individual and structural barriers that keep students from succeeding in higher education attainment. In continued pursuit of expanding access to quality and relentless reinvention, we at Champlain College just announced an undergraduate online tuition reduction of 50 percent. With bold vision and the right commitments from us as college and university leaders, digital learning will be put to work to expand success across our diverse institutional landscapes.

Burck Smith, CEO and founder of StraighterLine

While the R in a student’s return on investment is driven by the completion of a respected academic program, the I is the money paid to obtain the credential, the time taken to get the credential and the risk of payment without success. Despite a lower cost of delivery for online learning, the I for students at most colleges is the same or higher. Not only does this study confirm that delivering online courses is cheaper than delivering face-to-face courses, but this conclusion is supported by other market characteristics such as the profits enjoyed by colleges and their revenue-sharing partners and the continued decline in the cost of inputs -- software, bandwidth, content and instruction (as more adjuncts are used) -- while prices steadily rise. By not reducing prices to reflect a lower cost of delivery, powerful strategies to grow the student’s ROI are lost. However, some colleges and students are finding these strategies on their own.

Online providers do exist that improve the student’s I -- they just can’t be called colleges. For instance, StraighterLine’s general education courses are priced below those of most community colleges, and its free trial and subscription pricing enable faster-paced learning (if desired) and a lower financial risk for nonsuccess. In fact, a critical element of StraighterLine’s and Dallas County Community College District’s recently approved EQUIP program is its low-risk entry pathway for students. Prospective students will be required to successfully complete a free, online, credit-bearing course prior to enrollment and disbursement of federal financial aid. Successful students save money and are primed to succeed. Unsuccessful students have risked little and can be counseled to delivery formats that might be more appropriate.

Increasingly, colleges are also realizing that, when paired with their existing financial aid eligible models, educational formats that lower a student’s I can help increase enrollment conversion and student retention and persistence. For instance, colleges often can’t enroll or retain a student because of worries about the student’s academic readiness, lack of prerequisites, continued financial aid eligibility, inability to take selected general education courses and more. These students can be referred to a noncollege online provider whose lower I lets students solve these problems at substantially less cost and risk. These students return to the referring college and persist at rates greater than the rest of their cohort.

The use of these strategies by students and colleges will only increase as more colleges heed the advice of this report, build quality programs, enter the online learning market and create greater competition for students. For close to 20 years, the growth of online learning has been driven by its convenience. The next phase of growth for online will be driven the student’s I in ROI.

Terri Taylor Straut, director of actualization, ActionLeadershipGroup.com; co-author of the WICHE Cooperative for Educational Technology (WCET) 2017 report “Cost and Price of Distance Education”

The report highlights institutions that have made a commitment to redesign their online courses and programs to address the goals of decreasing the price students pay and improving student outcomes, as measured by retention and grades. We have had examples of such successful online implementations for years; Rio Salado is one of them.

The difficulty is in getting institutional buy-in to truly reinvent online courses, with these goals in mind. Many institutional leaders and governing boards have seen expanding online offerings as a way to increase enrollment, and therefore operating margins, without a systematic commitment to improving the students’ learning experiences. Those initiatives often result in higher fees and/or tuition costs to students.

Institutional leaders dedicated to increased student access through online courses and programs can learn from case studies in this report. Successful online programs are the result of strategic initiatives that start with an analysis of the current state of online learning at the institution. Faculty buy-in is crucial, and to attain it, faculty need professional development and adequate support for the technology. All of this requires financial investment to build institutional capability.

Those who believe that simply moving a course syllabus online and replicating the activities of the classroom suffices should tread lightly and scale back their expectations for success. The initiatives discussed did not merely dabble around the edges. Each one of them shows evidence of significant commitment to the creation of courses to meet the identified needs of their students. Each has employed technology to reduce the faculty burden of increased student interaction; each has invested in technology support for students and faculty.

Launching an effective online program requires a commitment to rethinking and retooling. It is not a quick fix to lagging on-campus enrollment, though it is often viewed this way. As reported last month by WCET, on-campus enrollment has declined by 4 percent, while online enrollment has increased by 13 percent since national Integrated Postsecondary Education Data System distance education data became available in 2012. These national trends may lead decision makers to believe that distance education is “low-hanging fruit” to address enrollment declines. If only it were so easy; institutions have been struggling with these challenges for over 20 years. When implemented as part of a strategic initiative to improve access and student learning, the case studies included here can inform institutions that are ready to make a serious commitment to their online initiatives.