Learning Science for All

Over the next 12 months, Carnegie Mellon University will give away a large portion of the learning science tools its researchers have spent the past decade developing.

The collection of tools, software and content, including underlying source code, will be made open so that any postsecondary or K-12 institution may freely use it.

Rather than making everything available at the same time, Carnegie Mellon will be releasing the collection in phases with accompanying guidance on how to conduct applied educational research using the tools.

The effort is being overseen by the Simon Initiative, a learning engineering enterprise at Carnegie Mellon that aims to measurably improve student learning outcomes through science. Guided experimentation with the tools will be encouraged through the Empirical Educator Project, a learning science initiative led by the higher education consultancy Mindwires Consulting.

“We’ve seen far too many open-source projects get thrown over the fence out into the world without adequate preparation,” said Norman Bier, executive director of the Simon Initiative.

Bier wants to take the time to prepare documentation that will make the content releases useful to other institutions. He also wants the tools, some of which are already available free, to become a more cohesive and integrated collection -- something he describes as a learning science ecosystem.

The full list of tools to be released as part of the collection has yet to be announced, but tools developed by Carnegie Mellon’s Open Learning Initiative are expected to be part of the package. The learning data and analytics platform LearnSphere has also been mentioned as part of the collection.

The release of the learning tools, which are the product of more than $100 million in research funding, is undoubtedly a generous gift to the academic community. But it also represents a shift in strategy for the university, said Paul Freedman, CEO and co-founder of Entangled Solutions, a company that focuses on innovation in education.

Carnegie Mellon has previously found commercial success spinning out adaptive learning tools into companies such as Carnegie Learning and Acrobatiq. But it seems the university has found it difficult to replicate this success without moving away from its research focus, said Freedman.

If Carnegie Mellon wants to foster a reputation for thought leadership in the space, making its tools open source is a smart move, said Freedman. “The direct commercial returns from the intellectual property aren’t as important to Carnegie Mellon as being seen as the intellectual leader of education technology,” he said.

Giving more people access to products, to test them and provide feedback, is a good thing for development, said Freedman. But the university won't be able to monetize its products the way it would if it were granting exclusive licenses to businesses, he said.

That is not to say that Carnegie Mellon can't make money by going open -- the university can still offer premium versions of its products, and it maintains control of licenses for commercial use on some tools, said Bier. There is also an opportunity for Carnegie Mellon to offer professional development services -- training others to use its tools and develop their own.

Without access to the code underlying these tools, many potential partners had been hesitant to use Carnegie Mellon’s products, said Bier. This is not unusual, said Freedman. Developers are often wary of building things on top of products that might not be supported in perpetuity. “If you don’t have access to the code base, there’s a lot of business risk,” said Freedman.

Kenneth Koedinger, professor of human-computer interaction and psychology at Carnegie Mellon, helped to co-found Carnegie Learning. Koedinger has been involved in many learning science projects at the university, including LearnSphere -- one of the tools that will be made open source. Though he has successfully commercialized tools in the past, Koedinger said he pushed for resources like LearnSphere, which he considers part of the learning science infrastructure, to be made open source.

“The market for these tools isn’t always clear,” said Koedinger. Ultimately, he wants the resources that Carnegie Mellon has created to be used by as many people as possible, and going open source will make this process “faster and better” than if they went down a commercial route, he said.

Learning science data are underused in course design, said Koedinger. Many instructors could drastically improve learning outcomes if they were more purposeful about the choices they make, he said. An analysis of the use of OLI textbooks at the Georgia Institute of Technology, for example, showed that students who complete assessments online perform six times better in exams than those who watched videos or read.

Room for Experimentation

Encouraging other institutions to take advantage of the analyses that Carnegie Mellon has performed and replicate them themselves is a big element of the university’s plans, said Koedinger. Carnegie Mellon will be working with the Empirical Educator Project to help institutions conduct tests using tools made freely available by Carnegie Mellon and other universities.

The Empirical Educator Project hopes to accelerate the pace of innovation in education through collaboration, said Michael Feldstein, director of the project. Over the next year, the project will be hosting “hackathons” where educators try to work through questions that will help them improve their practice. Among the questions: What is the best way to prove that a learning science approach is effective?

“We are trying to build a culture in which educators are always challenging their own assumptions about how they educate,” said Feldstein. The hackathons will be held as part of an initiative called EDwhy -- the "ED" standing for educational design.

Matthew Rascoff, associate vice provost for digital education and innovation at Duke University, hopes the Empirical Educator Project will start a “movement” of more evidence-based practices in higher education.

Duke will be contributing its own content for EEP members to work with -- a tool called WALTer that makes it easier for faculty members to navigate the institutional review board approval process needed to perform learning experiments on their own students. At Duke, the tool has helped faculty members to cut down approval times from months to 10 days.

The community that the EEP has started to build is remarkable because it will bring together not only university researchers and administrators, but also vendors and publishers. “It’s not often that you get to be part of a group like this where you’re not being sold to,” said Rascoff. “It’s not the sales team that’s participating, it’s the learning science team.”

The details of how participants in EDwhy will work together have yet to be determined, said Feldstein. EEP members will be meeting at Carnegie Mellon's campus in May to discuss how they can best take advantage of the trove of tools they have been invited to experiment with.

“The breadth and depth of Carnegie Mellon’s contribution represents a huge opportunity for us,” said Feldstein. “If only we can learn how to take advantage of it.”