Photo by Anna Shvets from Pexels

Introduction

Massive Open Online Courses (MOOCs) have been gaining traction over the last few years, with every major university launching their online channel and the advent of digital platforms, e.g. Coursera, Udemy, edX.

This adoption has been further accelerated by the Coronavirus pandemic where many schools / universities have been forced to switch to the online medium. So the online delivery of lectures has moved from a niche market to the mainstream product used by billions of people worldwide.

Innovation in the online education field has so far focused on improving the lecture content and making it more engaging for students, e.g., using gaze tracking to assess student attention [1], and gamification techniques to improve the content interactiveness [2].

With more and more people taking online courses, we focus on the notes (annotations) generated by these participants. As crowdsourced content, these notes have the potential to add a lot of value to both improving the original lecture content and aiding fellow students — by personalizing notes and establishing a notes marketplace. Unfortunately, the current online delivery platforms are not technically equipped to exploit the value add provided by student notes, at least not in an automated fashion.

Innovation

Focusing on student generated content (notes / annotations), in the context of online delivery of educational content; we propose 3 scenarios (illustrated in the figure below):

1. Crowdsourced Notes based Lecture Improvement: Improve lecture content by analyzing the notes captured by a diverse group of students (weak-strong students belonging to different demographics).
2. Notes Marketplace: Real-time offers of notes to weak students, or those who missed part of the lectures due to some distraction (someone entering the room, ringing the bell), or simply lack of attention. In the spirit of a marketplace, the note authors may also specify sharing policies, restricting or making their notes available to only a sub-group of students, or with variable pricing.
3. Notes personalization: Adapt Student A’s notes before sharing them with Student B. Such adaptation may be necessary to protect A’s privacy, or to make the notes more appealing to B according to his/her personal situation and preferences.

Crowdsourced Notes based Lecture Improvement

In this section, we outline how to use student generated content (notes, annotations) to improve lecture content.

For instance, consider a MOOC lecture that is delivered by a wordy (boring) Professor. As the students attend the lecture and make notes — the ones who understand are able to describe the points in a more concise manner than the Professor. These ‘descriptive’ notes are then used as the basis of annotation that are embedded in the lecture slides / video for use by future students.

A sample implementation would consist of the following steps:

1. Collect the notes made by students as they watch the online video content. Use Optical Character Recognition (OCR) and Natural Language Processing (NLP) based text extraction techniques [3] to align the timeline of the notes with the lecture content.
2. Classify students by grades, demographics, or groups that belong to certain ‘styles’ of learning. For instance, a comparison of the notes captured by two groups, e.g. best and worst — can be used to identify the different concepts that were understandable / confusing.
3. Take the timestamped notes for each group and use NLP based summarization techniques [4] to create a representative summary.
4. Embed the notes summary in the lesson video as a complementary artefact, at a time that is relevant, and then make it available to future ‘relevant’ students.

We address the “relevancy” aspects, in terms of deciding which (parts of) notes to offer to which students in the next two sections.

Notes Marketplace

We enable a notes marketplace by:

a) providing in-time delivery of (relevant portions of the) notes to students who need it — the CONSUMERS;

b) while allowing the note authors — the PRODUCERS — to control the students who can access their notes.

Recommending the relevant portions of notes to the (Consuming) students at a time they need can be performed by monitoring their gaze, or based on Wearables, Mobiles, IoT, etc. sensors to determine when they got distracted (e.g. someone entered the room, or their phone / doorbell rang); and their classification as a strong-weak student.

Similarly, the students authoring the notes (Producers) can also decide to restrict certain groups of students from accessing their notes based on access control policies.

Notes Personalization

The final scenario consists of personalizing the notes, including the lecture content, according to the profile and preferences of the consuming students — to make it more appealing / engaging for them.

For instance, let us consider an architecture professor is given a lesson that includes reference to the ‘Gothic Style’ of architecture.

1. Within the original course material either a green screen [5] or generic image is used.
2. At the start of the course a file is sent to each student specifying media requirements of the course, including an image of an ‘Gothic Style’ building.
3. Student’s photo library is searched and an appropriate image is found — the generic image in the original course material is changed to show the student’s own image.
4. As the (Consuming) student B is based in Lausanne, and an image of the newly opened Gothic Architecture Archives building has been previously captured by his Lifelogging camera; its image is embedded in the B’s personalized version of the course material/notes.

A similar enablement also applies when a ‘location’ revealing building image in the (Producing) student A’s notes (a building in Krakow, the home place of B) is replaced by the building in Lausanne, before sharing the material with B — to achieve both privacy preserving (from A’s perspective) and personalization (from B’s perspective) goals.

Sample technical implementations include:

a) Based on the learning / viewing pattern (inferred from gaze, Mobile, IoT Sensors, etc.) of the (Consuming, e.g. weak) student, emphasize/de-emphasize potions of the notes (including recorded lecture content) as follows:

• - Making sections of the image brighter/less bright
• - Colour change (make distracting object mundane)
• - Pixelate confusing material
• - Ken burns effect to encourage gaze to optimum position

More advanced personalization/adaptation techniques include:

b) From the Producer’s (notes author’s) point of view, this includes using OCR and NLP to remove privacy sensitive items from the notes that will be shared, e.g. names, location, personal situation — which they might have captured to improve the ‘recall factor’ of the notes.

c) Once the privacy preserving sections have been removed, further adaptation of the notes can be performed based on the Consuming student’s own content. This is enabled by specifying the media requirements of the notes / lectures in a way that the student’s own media matching the requirements can be found. Depending on the scenario (i) finding media that matches requirements, or (ii) identifying opportunities to capture new media that matches requirements. Finally, the identified media is embedded into the course material.

References

[1] Sharma, K., Giannakos, M. & Dillenbourg, P. Eye-tracking and artificial intelligence to enhance motivation and learning. Smart Learn. Environ. 7, 13 (2020). https://doi.org/10.1186/s40561-020-00122-x

[2] Borrás-Gené, O.; Martínez-Núñez, M.; Martín-Fernández, L. Enhancing Fun through Gamification to Improve Engagement in MOOC. Informatics 2019, 6, 28.

[3] Baligand, L., Biswas, D. Building an enterprise Natural Language Search Engine with ElasticSearch and Facebook’s DrQA. 10th Berlin Buzzwords Conference, Jun 2019

[4] A. P. Widyassari, et. al. Review of Automatic Text Summarization Techniques & Methods. Journal of King Saud University — Computer and Information Sciences, 2020 (link)

[5] Filming With Green Screen: Everything You Need To Know (link)