The Need for Greater Productivity through Online Learning, Further Thoughts

January 28, 2014

Thank you to Tony Bates for providing his perspective on productivity and online education, including part 1 on “Main Concepts and Principles” and Part 2 on “Identifying Promising Areas of Productivity for Online Learning.” 

We asked five experts to give us their reaction to what Tony wrote.  They also give their own thoughts on the research on this issue and alternative views on key components of productivity in higher education:

• Katrina Meyer, Professor of Higher and Adult Education, University of Memphis:  “Focus on Students and Faculty; More Research is Needed”.
• Carol Twigg, President and CEO of the National Center for Academic Transformation: “NCAT Has the Research to Show Quality and Productivity Gains”.
• Thomas Cavanagh, Associate Vice President, Distributed Learning, University of Central Florida: “A Plurality for Productivity”.
• Dennis Jones, President, National Center for Higher Education Management Systems: ” Focus on the Program (Instead of the Course) for Productivity Gains”.
• Phil Hill, Co-founder of MindWires Consulting and blogger at e-Literate:  “Watch California for the Next Steps in Productivity”.

Thank you to each of them for sharing their thoughts and furthering the discussion.

Russ Poulin, WCET

Focus on Students and Faculty; More Research is Needed

Katrina Meyer

Professor of Higher and Adult Education
University of Memphis
kmeyer@memphis.edu

I agree with Tony’s thoughts on achieving greater productivity in online learning, but would like to add three related thoughts for all of us in online education to ponder.

Focus on individual student needs and how to help them learn.
So many of the current developments in online learning (such as MOOCs and the stress on content) seem to be based on an assumption that we are dealing with an ideal student: the self-disciplined, relatively able student who is motivated to learn. Give them content or a MOOC and they will likely teach themselves. Certainly this applies to a certain portion of students, but by no means all. Many students – be they high school graduates or adults needing further education — arrive at higher education institutions with inadequate preparations that may include poor skills or work habits. They are often new to online learning, expect the same passive learning approaches as they experienced in earlier educational settings, and lack confidence in themselves. Despite the prevalence of the perception that these students are intuitive technology users, many of them are not. We really need a lot more focus on these students’ needs, on both how to help them learn and do so while using institutional resources productively. I would argue that we need more focus on these kinds of students as we continue discussing what is quality education, how we can be more productive, and how to measure it.

Identify when faculty skills are necessary.
My second thought relates to the role of faculty. To be productive as online learning programs, we need to use faculty time and skills more productively. We already know that technology can ably replace faculty when it comes to delivering content or testing knowledge. But we also need to identify those times in the learning process that faculty skills are necessary, such as in identifying students’ thinking errors and solutions to those errors, which pedagogical approaches work best for students with different needs, and what content might be most motivating to a particular student, helpful to sharpen their thinking, or needed to broaden their thinking. Faculty are often able to identify when a student needs reassurance that he or she can do the work of the class, a listener who can perhaps offer alternative solutions, as well as a compliment when earned. Our search for productivity needs to identify these necessary faculty roles and ensure they still occur in the more productive online setting of the future. I suspect that we will begin to think of faculty as critical “learner support,” too.

In my research on faculty who teach online, I have been impressed by the number and variety of faculty who care deeply about their students and work hard at finding ways to help them learn and to do so in a way that is productive for them and their institution. I call this “student learning productivity” which places student learning at the center of productivity efforts of faculty. I do not mean to diminish other types of productivity efforts, but wish to point out that faculty may be especially useful in tackling the challenges of helping students learn more and to do so more efficiently.

We need more research.
Lastly, we do need more research, as Tony mentions. We need research that compares pedagogies rather than focuses on a test of one pedagogical approach or one program. We need research that incorporates an assessment of cost or productivity as well as an assessment of pedagogies or programs. And we need research that identifies which kinds of learner support matters most to learning and whether the cost of all kinds of support structures is worthwhile.

Thank you for allowing me to add these thoughts to Tony’s piece and I hope that a focus on productivity becomes a regular focus of the efforts of WCET members!

 

NCAT Has the Research to Show Quality and Productivity Gains

Carol Twigg
President and CEO
The National Center for Academic Transformation
ctwigg@theNCAT.org

I’ve been making the argument that technology is key to increasing productivity in higher education since my first published article on this subject, “Improving Productivity in Higher Education: The Need for a Paradigm Shift” in 1992.

Since 1999, in partnership with more than 200 colleges and universities, the National Center for Academic Transformation (NCAT) has proven that it is possible to improve quality and reduce cost (increase productivity) in higher education through course redesign. Course redesign is not just about putting courses online. It is about rethinking the way we deliver instruction in light of the possibilities that new technology offers.

• NCAT and its partner colleges and universities have initiated 195 redesign projects, 80% of which were completed.
• Of the 156 completed projects, 72% improved student learning outcomes; 28% showed learning equivalent to traditional formats.
• Of the 156 completed projects, 153 reduced their costs by 34% on average (ranging from 4% to 81%).
• Institutions participating in Changing the Equation, an NCAT program focused on developmental math at community colleges, reduced their costs by 20% on average; all other redesigns reduced their costs by 37% on average.
• Collectively, the 253 courses that have been redesigned enroll about 250,000 students annually.

Quality Comparisons
NCAT requires each participating institution to conduct a rigorous evaluation of the impact of the redesign on learning outcomes as measured by student performance and achievement. All NCAT redesign projects compare student learning outcomes in the traditional format with those achieved in the redesigned format by 1) running parallel sections of the course in the two formats or 2) comparing baseline data from an offering of the traditional course to a later offering of the redesigned course, looking at differences in outcomes in the “before and after.”  The four measurement methods used to assess student learning include 1) comparisons of common final exams, 2) comparisons of common content items selected from exams, 3) comparisons of pre- and post-tests, and 4) comparisons of student work using common rubrics.

Cost Comparisons
NCAT requires each participating institution to establish a team of faculty and staff who will conduct the redesign. Each team analyzes and documents “before and after” course costs using activity-based costing. NCAT developed a spreadsheet-based cost planning tool (CPT) that supports institutions in this process. By completing the CPT, participants are able to 1) determine all personnel costs; 2) identify the tasks associated with preparing and offering the course in the traditional format and determine how much time each type of personnel spends on each of the tasks; and, 3) identify the tasks associated with preparing and offering the course in the redesigned format and determine how much time each type of personnel spends on each of the tasks. The CPT then automatically calculates the cost of both formats and converts the data to a comparable cost-per-student measure. At the beginning of each project, baseline cost data for the traditional course and projected redesigned course costs are collected; actual redesigned course costs are collected at the end. Completing the CPT allows faculty members to consider changes in specific instructional tasks, make decisions about how to use technology (or not) for specific tasks, visualize duplicative or unnecessary effort and complete a cost/benefit analysis regarding the right type of personnel for each instructional task.

National Research Council Report
In 2012, the National Research Council published a 192-page report, Improving Measurement of Productivity in Higher Education suggesting a new set of metrics that would allow for a sector-wide look at productivity. Sponsored by Lumina Foundation for Education, the report was produced by a National Academies panel chaired by Teresa Sullivan, president of the University of Virginia. NCAT’s work is featured in the report as a contemporary example of how instructional productivity can be increased.

The report presents an analytically well-defined concept of productivity in higher education and recommends empirically valid and operationally practical guidelines for measuring it. Productivity, as the report’s authors define it, would factor in both credit hours completed and degrees awarded compared to labor costs and other expenses. By accounting for both degrees and credit hours, the model would reward institutions for graduating students without penalizing them for having large numbers of part-time students.

A Plurality for Productivity

Thomas Cavanagh
Associate Vice President, Distributed Learning
University of Central Florida
cavanagh@ucf.edu

The issue of productivity is becoming increasingly important. As policymakers place growing pressure on postsecondary institutions to improve efficiency, colleges and universities are being forced to explore new technology-driven models. Simply “putting a course (or a program) online” is no longer the bold innovation it once was. The higher education community must continue to innovate, leveraging technology in the service of the institutional mission.

But what does “productivity” mean? How do we define it? More students completing degrees more quickly? A number of new models have been hyped recently as the solutions to improved productivity. Certainly we have all heard plenty about MOOCs. But thus far they have not reduced anyone’s time to degree. Competency-based education is gaining traction as a model for productivity—let students move as quickly as they want/can through a program. But such an approach isn’t right for everyone. Adaptive learning holds significant promise. However, while there may be significant pedagogical advantages to adaptive learning, whether such an approach can truly improve productivity remains to be seen.

Different types of students must be served in different ways. Many in the postsecondary world, including Clayton Christensen and Paul LeBlanc, speak about the disaggregation—or unbundling—of higher education. As we look at the kinds of students a single institution might be educating in the future, it is an extremely heterogeneous group. For the traditional 18 year old first-time-in-college student, we may offer a comprehensive residential campus experience, complete with dorms, fitness center, and Saturday afternoon football games. For community-college transfer students, a mixture of state college, regional campus, main campus, and online courses may be what is necessary and preferred. For these students, the on-campus residential experience has been “unbundled” from the delivery of instruction. Such students neither need nor want it. For other non-traditional students, including working professionals and active-duty military, a fully online option may be the only choice for educational access. For others, perhaps it is a low residency program, or a competency-based program, or a collection of credits from a variety of institutions (or even MOOCs) brought to your school to complete a degree.

What we need is a plurality of choices—a continuum of options from which students may select. We need technology-enabled structures that support and encourage students in the most effective way to achieve their own individual success. If we can create this educational ecosystem of choice we will empower students to choose the program or combination of programs that best fits their own unique needs and moves them along the most productive path to academic success.

Focus on the Program (Instead of the Course) for Productivity Gains

Dennis Jones
President
National Center for Higher Education Management Systems
dennis@nchems.org

Tony correctly identifies both the forces pressing for increased productivity in higher education and the role that online learning can play in achieving that productivity.  He also points out several different ways in which that contribution could be made.  I would like to build on this foundation and add an additional perspective.

Almost all the work on productivity enhancement through use of online learning has focused on the course as the unit of analysis.  Additional opportunities for productivity gains—accomplished while maintaining or even improving quality—can be found if the program, rather than the course, becomes the focus.  Adopting this perspective allows:

1. Defining the degree as the output of the process.  This both simplifies the measurement and aligns outputs with statements of goals which now almost uniformly include education attainment as a priority.  This doesn’t eliminate a concern for quality, but it does make it an issue of parallel concern rather than one that has to be addressed as a central element.
2. Broadening the arena of investigation by opening the question of how to use online learning to produce a larger number of degrees (the bang) for the buck.
3. Most important, it allows a broader array of choices regarding assignment of human assets to learning and learning support activities.  More specifically, it allows teaching foundational courses with heavy reliance on technology and less costly personnel while using senior faculty in those more advanced classes designed to encourage much deeper learning.  This makes the personnel utilization decision a both/and decision rather than an either/or choice.

At the center, what I’m arguing is that it would be helpful to raise the discussion about use of online learning to a strategic level by asking how it can cost-effectively contribute to goal attainment.  Incidentally, this focus raises the question about how technology and less costly personnel can be deployed to provide support services that aren’t necessarily in service to completion of a single course—the role of student coaches that help students over all the hurdles they face, not just those associated with a single course.

Watch California for the Next Steps in Productivity

Phil Hill
Co-founder of MindWires Consulting and blogger at e-Literate
philonedtech@gmail.com
Twitter: @philonedtech

Tony’s magnum opus on productivity issues from online learning is invaluable, especially if seen as marking the transition from talking about the nebulous concept of ‘whether we can save money by using online education’ to talking about ‘how we can use online education as one tool to help us become more productive in our educational mission’.

Consider California as an example. 2014 is shaping up to be the year where the three public systems and the faculty take ownership in figuring out how to apply online education to ensure students have access to the courses they need for graduation or transfer. While there has been a tendency to view online education as a magical revenue enhancement, the key initiatives (Online Education Initiative, CalState Online & CourseMatch, and UC Online) are increasingly seen as investments in productivity, including $37 million this year in additional state funding, with a real focus on combining the resources of institutions across each system. These changes align well with the concept of ‘increasing access to higher education’. Based on the discussions at the recent 20 Million Minds Foundation Evolve conference, the key initiatives are also focused on ‘course design based on sound pedagogical principles’ and ‘learner support’ as key principles for the upcoming projects.

The California initiatives are based on good intentions and have a more mature understanding of productivity, but all of them are in their infancy. The concepts described in Tony’s posts will likely be seen in practice – for both positive and negative effect – over the coming 2 – 3 years in California. This is a very important topic both in state policy and in systemwide implementation.