Knewton Returns, With New Pitch

Knewton, a pioneer in adaptive learning technology, will launch its first direct product for higher education in January. Previously the company provided the technology behind several other companies' adaptive learning products, but now Knewton is changing its strategy and selling its product to colleges directly.

The name of Knewton’s proprietary offering hasn't been announced, but it has been piloted at more than 150 institutions, the company said. The product, described as a complete textbook replacement, began piloting early this year and was tested in beta in 2016. Using open education resources, Knewton offers courseware that both instructs and assesses students using adaptive learning technology.

In data shared with Inside Higher Ed and now published online, Knewton presented results from a sample of more than 10,000 students who used the Knewton courseware and completed over 130,000 assignments (defined as "instruction and assessment in Knewton tied to course learning objectives") so far this year. The data also look at the correlation between successful completion of these assignments and performance in class quizzes set by instructors. The sample includes students who used the Knewton product for courses in mathematics, chemistry, statistics and economics. The students were told to complete the Knewton assignments as homework outside class, completion of which counted toward their grades.

In adaptive learning, students receive information that they are then tested on. The answers the students give to these questions determine the next instruction they receive. Those who demonstrate knowledge advance to the next topic, while those who don't get customized additional instruction until they succeed.

The Knewton data show that 87 percent of the time, students successfully completed the assignments. Among students who initially struggled to complete an assignment, 82 percent eventually did so -- an achievement referred to as “mastery."

In addition to a high level of assignment completion, Knewton said their product positively impacted performance in quizzes set by instructors. Students who completed less than 25 percent of their assignments achieved an average class quiz score of 55 percent, compared with 81 percent for students who completed more than 75 percent of the assignments.

Among struggling students (defined as students who started in the bottom 25 percent of the class) those who mastered more than 75 percent of the learning objectives achieved average class quiz scores of 70 percent, compared with 40 percent for those who failed to complete more than a quarter of the assignments. The data suggest that among those who completed the majority of their class assignments, the attainment gap between those who started at the top of the class and those who started at the bottom is closed significantly.

Knewton did not supply a full list of institutions that tested the product, but named a sampling of institutions currently using the Knewton course product, including: California State University at Los Angeles, Kellogg Community College, National Louis University, Central Oregon Community College, Park University, Collin College, Eastern New Mexico University, Hunter College and Front Range Community College.

Eyob Demeke, director of developmental mathematics at California State University at Los Angeles, said that his institution started using Knewton last summer as part of the trial. He said that so far, the reaction had been positive from both students and instructors, with courses reaching “impressive pass rates” in the 80s. Demeke noted, however, that as these courses had not been previously offered at scale, he did not have the historical data to compare results before and after using the Knewton product.

In institutional data shared by Demeke for two math courses using Knewton over two sessions, 69 percent of students rated their experience of the product as “good” or “very good.” Just under 10 percent of students rated their experience as “bad” or “very bad.” The rest described their experience as “neutral.”

A selection of student comments were included with the data from California State L.A., the majority positive, with some students saying the product had helped them to understand topics they had previously had difficulty grasping. Not everyone was convinced, however, with one student commenting that “having to pay to do homework is a rip-off.” Another expressed frustration at the amount of time it took to complete the assignments.

Asked whether his institution intended to keep using Knewton beyond this semester, Demeke said that it had not yet been decided. He noted the California State University system was considering free alternatives such as EdReady, a personalized math study program managed by the nonprofit NROC project.

Andrew Moore, assistant professor and math content lead at National Louis University, said that he had also had a positive experience with Knewton, which his institution started using as a supplemental instruction tool in January. He praised the ability of the product to highlight students who are struggling, saying that his institution had used this to intervene and offer extra support.

“Over all it looks like our current class are going to be performing at a higher rate than our students last year,” said Moore. As the statistics course using the Knewton product is a relatively new, Moore said he didn’t yet know whether this apparent increase in performance is linked to Knewton, or to faculty gaining greater experience of teaching the course. He said he would be reviewing the data closely over the winter break.

Illya Bomash, head of data science at Knewton, acknowledged that evidence of the efficacy of the product is limited by the lack of a control group. However, Bomash maintained the data were sufficient to suggest a causal effect between successful completion of assignments and high attainment in class. He said the results of the data were not particularly surprising, but a “confirmation” that the Knewton approach works.

Karen Vignare, executive director of the Personalized Learning Consortium at the Association of Public and Land-grant Universities, said that the initial data from Knewton “looks good” but “there’s a lot more you want to know.” For example, Vignare questioned who and what exactly defines mastery -- “is it something the professor can set?” she asked. Vignare said she would also like to see more demographic information on the students, and a clear comparison of the levels of understanding before using the Knewton product and after.

Michael Feldstein, a partner at MindWires Consulting, and co-publisher of the e-Literate blog, said that the data demonstrate that students who are taught by instructors with knowledge of mastery learning methods, and who use Knewton-based products that support those methods, tend to achieve the learning objectives for the course and complete the course. “That’s all great,” said Feldstein. “Unfortunately, it’s also utterly generic. Knewton has essentially released data showing that mastery can be effective, which we have known for a long time.” (MindWires has done consulting for some textbook publishers that now have their own adaptive platforms.)

“I don’t see anything in what they’ve released that tells us anything new, and I certainly don’t see anything that makes a specific case for Knewton over the many other mastery learning-based products on the market (whether or not they have whizzy adaptive algorithms),” said Feldstein. He added that in order to gain credibility, Knewton “will need to release original findings in a fully fleshed out and peer-reviewable paper, preferably accompanied by enough of the source data that their work can be checked.”

Brian Kibby, CEO of Knewton, said that he felt confident that Knewton’s product would stand out from other offers on the market. “We are the only ones in that world that do what we do,” said Kibby. “That is, use openly available content in partnership with our powerful personalized learning engine.” Kibby added that a core difference between the Knewton product, and others on the market, is that every student is treated as an individual. “Some of the technology groups learners in pockets,” said Kibby. “We don’t. We approach every learner as a unique learner.”

However, several companies contacted by Inside Higher Ed, including McGraw-Hill Education and Pearson, said that their adaptive learning technology also treats students as unique learners, challenging Kibby's claim.

Heather Shelstad, vice president of marketing for Knewton, said that while most adaptive learning companies would agree that every student is unique, Knewton's latest product is different in that it applies this concept of personalized learning to "all elements of a course," including course content, homework assignments and testing.

"Every interaction a student makes in Knewton impacts the complete learning experience, including instructional content and assessments, to dynamically support the most effective path towards assignment completion and concept mastery," said Shelstad. She explained that there were three general types of learning technology being used in higher education today -- digital homework platforms without adaptive technology, digital homework platforms with add-on adaptive technology, and fully integrated adaptive courseware. Knewton, she said, falls in the latter category. Knewton "continuously diagnoses and evaluates a student's progress toward a concept mastery and assignment completion dynamically and in real time. No two students are ever on the same path," she said.

Kibby, who joined the company as CEO in July and previously held executive roles at McGraw-Hill Education and Pearson, said that Knewton’s previous strategy of leasing its technology to other companies “was misguided” and “flawed from the beginning.” Knewton previously powered Pearson's MyLab and Mastering suite, but Pearson this year decided to end this partnership and develop its own technology.

Erin Joyner, senior vice president of higher education product at Cengage, said that company had previously partnered with Knewton, but decided to go in a different direction after the pilots "didn't produce the results we'd hoped for." Cengage has since partnered with Cerego to offer an adaptive learning product for psychology courses, as well as in-house-developed product called Adaptive Test Prep.

While many partners have been commercially successful with platforms built on Knewton’s technology, none have fully taken advantage of its power, said Kibby. As Knewton has the deepest understanding of its own technology, it is in the best position to deliver it directly to faculty and students, he added. While scaling up Knewton’s proprietary product will now be the company’s main focus, Kibby said the company would maintain its existing commercial partnerships where they continue to make sense for both parties.

Looking forward, Kibby has set some ambitious sales targets. He said that within three years he expects every college and university in the country to be using a Knewton product. His vision is for global expansion, not only in higher education, but also in K-12 -- an area where Kibby foresees a great deal of growth for the company. “When you combine the impact of our product with the affordable price point, how can you not look at it?” said Kibby. He added that early feedback for the product has been “tremendous” and he fully expects the product to “scale up and take off.”

Adam Newman, managing partner of Tyton Partners, an education consultancy firm, said that one of the challenges facing Knewton would be reintroducing it to the market as a courseware provider. Given that Knewton was once known as a test prep company, Newman said that many people may “have impressions of who or what Knewton is that may not be what Knewton is today.” While Kibby’s sales target is a “good stretch goal,” Newman said it was not one he would bet on Knewton reaching, despite Kibby having the background to “understand what it would take to be in every college and university in the U.S.” As for Knewton’s shift in strategy, Newman said launching a new product under a new name could be a good move for the company. “There’s been a lot of hype relative to what’s been delivered historically,” said Newman. “The ability to create a new identity would not necessarily be a bad thing in the postsecondary marketplace,” he said.

Affordability is a key issue in a market where prices are being aggressively driven down by the competition, said Vignare. She said the Knewton product did indeed sound affordable at $44 per student per course for two years’ access or $9.95 per month (plus tax). The product is currently available for 25 courses, mostly concentrated in the areas of mathematics and science, but with plans to expand to all core areas at an introductory level. The basis for the content that is taught and assessed in the Knewton product comes primarily from OpenStax, a nonprofit OER textbook provider, with which Knewton announced a partnership in May 2016.

James Wiley, a principal analyst at Eduventures, said that he thought Knewton’s new direct-to-customer approach made sense, especially as some of Knewton’s former partners, including Pearson, have stated intentions to develop their own adaptive learning capability in-house. “I always think a B2B approach is risky, as your partner could simply learn your technology and develop it for themselves,” he said.

An early leader in the adaptive learning market, Knewton has struggled to capitalize on its head start, despite bold statements about the power of its technology from previous leadership, said Feldstein. “The good news here is that the company seems to be moving away from former CEO Jose Ferreira’s deeply unfortunate characterization of the product as “a robot tutor in the sky that can semi-read your mind,” he said. Wiley, of Eduventures, said that it seemed the original excitement around Knewton “has somewhat died down. Our clients, for example, don’t really ask us about it much.”

Two of the biggest hurdles Knewton will need to overcome are a lack of awareness, and inertia, said Kibby. To overcome this, Kibby said that he has built a strong sales team with experience in education, who will be spending a lot of time on campuses. Establishing these campus relationships will likely be key to Knewton’s success, said Vignare, as other companies will already have strong ties with institutions.

As awareness of adaptive learning technology grows, Vignare warned that institutions would start to ask vendors tougher questions. Wiley agreed, saying that he expected many institutions to want to be able to customize every aspect of the technology. “Institutions we’ve spoken to, for example, want to have full access to the content, assessment and sequencing of a given tool. As you can imagine, vendors in this space may be loath to provide this for fear that allowing this may reduce their products to just a configurable set of algorithms,” he said.

Vignare warned that using adaptive learning technology successfully requires work. “You have to integrate it into your pedagogy. It can’t just be a system that sits alongside your course,” she said.