Research on "What Makes a Great MOOC"

December 18, 2013

Thank you to Colt Alton of Ednak for conducting this interview with Panagiotis Adamopoulos on his research regarding “What Makes a Great MOOC.” 

Adamopoulos is a PhD Candidate of Information Systems at New York University’s Leonard N. Stern School of Business. He has graduated with honors from the Department of Management Science and Technology of the Athens University of Economics and Business and worked as a Business Intelligent engineer and consultant for two years before pursuing his doctoral studies. His research interests focus on recommender systems, machine learning, and the phenomenon of MOOCs. His research methodology combines econometrics and machine learning techniques building both explanatory and predictive models.

Colt Alton is the founder of Ednak, a discovery engine that tracks trends, innovations, jobs, and events in education technology.

= = = = = = = = = =

Alton: The New York Times dubbed 2012 “The Year of the MOOC.”  How has 2013 responded to that? What would you call this past year?

Adamopoulos: 2013 has definitely been a good year for MOOCs. Even though we haven’t yet experienced the disruption of higher education that many people envisioned and expected, we observe a sustaining but revolutionary innovation, enabled and accelerated by technology. In 2013, many prestigious universities from different parts of the globe started offering MOOCs in a wide range of topics through either the existing platforms or even their own initiatives. Besides, we see many of these players experimenting with different strategies while diverse business models are emerging. At the same time, many providers are working out the kinks of MOOCs focusing on transforming higher education in a way that will be good for our society. Another important milestone for MOOCs in 2013 was reached when the American Council on Education evaluated and recommended specific MOOCs for college credit. Thus, if I had to characterize 2013 with respect to MOOCs, I would probably dub 2013 as the year of MOOCs going mainstream.

Alton: In the introduction of your report, you mention that there is still plenty of room for improvement as far as MOOCs addressing the needs of students. What are the main take-aways from your research on MOOCs?

Adamopoulos: MOOCs are a great advancement of our era alleviating most of the economic and location barriers of traditional education and offering high quality content to any interested students around the globe. Because of the significance of their impact and the importance of the opportunities they offer, MOOC providers, instructors, and educational institutions should carefully engineer the courses they offer. Moreover, given their massive scale, designing MOOCs in the optimal way is of crucial importance and this research makes a step toward this direction. We identify some of their components that should be further improved, such as certifications and discussion forums, and we also propose some feasible and very simple solutions to these problems. Besides, we offer specific guidelines about which courses should be calendar-based and which self-paced, the ideal workload and length for different types of courses, the format of assignments and exams, etc.

Alton: Were there any aspects of your findings that surprised you? Can you elaborate on what were the biggest surprises from your research?

Adamopoulos: The biggest surprise to me was that the reputation of the university that offers the course doesn’t play a significant role in student retention in the current generation of MOOCs. The reputation of the university might be important in attracting more students but the actual content of the course and the professor that delivers it constitute the main factors that affect the decision of the students to complete a course or not.

Another surprising result is the impact of peer assessment on student engagement. There is a lot of discussion nowadays about fully automating the grading process and further refining the existing technological solutions. However, we tend to forget that peer assessment is also an important part of the learning experience. It helps the students better understand an assignment and, more importantly, offers a glimpse into how others are solving the exact same problem. Hence, apart from developing better solutions for automated feedback, we should also work on finding better ways to pair students and get the most out of each assignment.

Alton: What areas of MOOCs did your research find to be over-hyped and what areas are under-valued?

Adamopoulos: The most over-hyped aspect of the current generation of MOOCs is probably the certifications. In principle, certifications verify that a certain person is adequately qualified to perform a job or task. However, MOOC certifications, at least their current version,  provide neither knowledge verification nor user identification. Thus, they are of limited usefulness to the users. However, there are certain ways in which we can solve this important problem.

On the other hand, one aspect on which we should put a lot of emphasis is the design of a MOOC and the delivery of the content. Not only professors are of extreme importance but also the way that a course is structured; how the educational material is allocated over the length of a course, or even what type of assignments and exams are offered have significant impact on the active engagement of students. Interestingly, we could also personalize each course and offer a better learning experience to each one of the participants.

Moreover, beyond specific courses, an important under-valued aspect of MOOCs is the acquisition of unexpected knowledge. Apart from helping MOOC users master popular topics such as data science, we have a tremendous opportunity to help them further explore their interests and talents by acquiring for free valuable and high quality knowledge about other areas and domains that they weren’t even aware of. By carefully offering them diverse information, we have the opportunity to burst some of the “filter bubbles” in higher education.

Alton: Panos, let me ask you to elaborate on the topic of MOOC certificates. The report states the current form of MOOC certificates provided limited usefulness to students. Is there a model that would be more useful for students and those looking to verify competence in a particular MOOC subject matter? How can MOOC providers make this aspect of MOOCs more compelling?

Adamopoulos: We have already seen Coursera implementing the “signature track”, a new type of certification aiming to alleviate some of the weaknesses of the previous generation of certifications, especially the issue of student identification. Apart from that, students interested in particular subjects, such as programming, nowadays, have the chance to create their own portfolios in order to demonstrate their abilities. One way to further enhance this aspect of MOOCs would be the providers or other third-party institutions to administer standardized tests similar to the professional certifications in various industries. This would also offer to MOOC participants a path to employment since it will allow companies to hire employees based on specific skills acquired through MOOCs.

Alton: Your research eluded to an interesting point related to discussion forums in MOOCs. The results suggest forums may in fact serve as a negative effect on student satisfaction and may contribute to attrition. Can you elaborate on this further?

Adamopoulos: Because of the plethora of participants in MOOCs, discussion forums have been used in order to facilitate the communication with the instructors and among students. We have found that discussion forums are attractive to students and actually influence their decision whether to enroll in a course and complete a part of it. However, they don’t contribute to further engaging the students and helping them successfully complete a course.

One of the main reasons is that many MOOCs are extremely imbalanced in terms of the background of the participants. Because of the open nature of MOOCs, some participants are much more advanced than others and hence we might need to create different sections and clusters in the forums for students to benefit more from the communication that takes place in forums. Also, another reason is that the right norms have not been established yet. For instance, initially there were some “bullying” incidents with a small but vocal number of participants openly supporting that specific courses are only for the elite of students and not everybody.

Fortunately, the MOOC providers are well aware of such incidents. One solution that some providers have been implementing the last month is to assign a larger number of teaching assistants to the forums in order to better help students and moderate the discussion. The introduction of wikis can also supplement and enhance the discussion forums by providing a reference for codified knowledge and separating the learning process from the miscellaneous communication among students. Local study groups can also help in further engaging students.

Alton: Some have suggested MOOCs could serve as a disruptor to the textbook industry. How do you see the relationship between MOOCs and textbooks unfolding in the future?

Adamopoulos: Even though the building block of most MOOCs is video lectures, it’s very hard for a series of videos to reach both the breadth and depth of a good textbook. This is one of the main reasons that many MOOCs also recommend the use of a combining textbook. Actually, we can also observe that MOOC students are still benefited by using textbooks and there is a strong complementarity effect between textbooks and MOOCs. Hence, even though textbooks might change format or be accompanied with multimedia such as videos, I don’t think that this industry will be disrupted because of the MOOCs.

Alton: The findings of your research suggests the content of a MOOC is becoming a more important factor in determining student persistence than the institution’s reputation. Could this change the landscape of how institutions offer MOOCs? Is there an opening for new higher ed players to emerge?

Adamopoulos: The institution’s reputation in traditional education does not affect student persistence in MOOCs as much as the actual content of a MOOC or the instructors do. This indicates both that there is an opening for new higher ed players and that institutions offering MOOCs now have the chance to build a better reputation specific to online learning. We should consider though that the MOOCs with the best content are usually associated with the institutions that already enjoy the highest reputation in traditional education. Moreover, some of these institutions are building on their extensive experience in distance learning and e-learning environments that have acquired based on related initiatives in the past. Thus, new players have to offer better content in order to succeed and build their own reputation.

Another strategy for new players would be to offer accreditation for MOOCs and more useful certificates. Such certifications could also change the role of institution’s reputation.

Nevertheless, it is very hard to predict with confidence how the higher education industry will evolve during the next years but I would expect the big players to remain the same as in traditional education and the second tier institutions to start facing price pressure and increasing competition from new players.

Alton: Time to dust off your crystal ball…where are we headed with MOOCs? What predictions would you venture to make about MOOCs in 2014?

Adamopoulos: I expect more universities to enter this new market of MOOCs and start offering courses online either through an existing platform or by starting their own ventures. In the meantime, existing providers will offer better courses through the use of analytics and tapping into the data they have been collecting. In addition, I expect technological companies to get involved and offer courses on the new technologies that they are building in an effort to expand and enhance their talent pool. 2014 might also be the year for highly specialized educational programs and institutions focused on special verticals, such as Data Science, to emerge.

Colt Alton
ca@ednak.com

Panagiotis Adamopoulos
padamopo@stern.nyu.edu