A Population Approach to Evaluating Edtech?

What edtech conferences do you play to coming year?

 

What will we talk about during the edtech 2015-2016 season of meetings? Will we talk about the right things?

 

I’ve been going to these conferences for years, and I’m starting to wonder why. 

 

Each year the edtech conferences get bigger, the educational technology gets more impressive, and the number of amazingly innovative programs and initiatives at the intersection of learning and technology seems to increase.  

 

And yet, year after year all this educational technology seems to have little or no impact on core measures of nation’s postsecondary health.

 

If educational technology is so wonderful how come we have don’t see measurable and large scale improvements in costs, access, or quality?  

 

I’m starting to worry that our edtech profession has gone down the same road as much of our medical profession. Lots of specialization and local progress, but little or no discernible impact on population health. 

 

About a quarter of all healthcare spending in the U.S. occurs during the last months of life.  Healthcare spending is running at about 17.4% of our GDP, equal to about $3 trillion a year.   Despite all this money spent on healthcare, spending that is considerably higher than the rest of the world, the U.S. still lags other wealthy countries on almost every health outcome measure. 

 

My argument is that our edtech community needs to shift to a population based approach to evaluating our investments and our programs. Rather than talking about localized measures of quality improvement or cost reductions, we should instead attempt to measure the impact that educational technology has on the larger postsecondary system.  

 

Thinking at a population level is not something that comes naturally to our edtech profession. 

 

We are not very clear about which outcomes we should be measuring.  What is the appropriate statistic to judge the health of the higher education ecosystem? Is it enrollments? Total or percentages? Graduation rates? Tuition? Student costs? Employability? 

 

Or should we be measuring things like the percentage of faculty who are adjunct, contingent, or employed full-time? Do we care more about input measures - such as the average student-to-faculty ratio or the dollars spent on instruction - or outcomes such as graduation rates and student debt?

 

Moving to a population based orientation for thinking about educational technology will change our frames of reference. We may be less impressed with higher ed innovations if they do not show up in measures of postsecondary productivity. We may spend more time learning about higher education economics and public policy, and less time discussing the latest technology platform or new pedagogical technique.

 

Where is the right place to start If we were to take a population approach to understanding the impact of educational technology on the higher education system?

 

What are the common outcome measures that we should agree on?