AI Article – Concept Mastery

“Learn at your own pace” is what all schools claim as their motto. Honestly, it’s not possible to achieve it because of the disproportionate student-teacher ratio and as all students are given approximately the same amount of time to learn and the same set of instructions. This is what we refer to as traditional learning. Now, to remedy this situation, we need Mastery Learning. Mastery learning (or, as it was initially called, “learning for mastery“) is an instructional strategy and educational philosophy, first formally proposed by Benjamin Bloom in 1968. Mastery learning maintains that students must achieve a level of mastery in prerequisite knowledge before moving forward to learn subsequent information [1]. Basically, Mastery learning aims to identify the gaps, resolve the gaps and then allow the student to move forward to the next level as long as it takes. 

Embibe is an AI-powered platform that aims to provide personalised learning to each and every student around the world. Hence, we did our homework and we landed on mastery learning. In order to implement mastery learning, capturing the knowledge state on all concepts is essential. Capturing the knowledge state is accomplished by monitoring the students’ interactions with the platform at the concept level. These interactions range from watching videos, practising questions, taking tests, and even viewing test feedback. Modelling these interactions to find if a student has mastered a concept is referred to as Concept Mastery. 

Concept Mastery is inherently a complex task that requires modelling how humans acquire knowledge. Hence, we keep track of all interactions in a student’s learning journey on the platform. We do so in order to identify any event which might cause learning and the events that can be evidence of a student’s learning. Another major hurdle for Concept Mastery is data sparsity. We cannot expect any student to make attempts on all 65000+ concepts of our Knowledge Graph so that we might be able to get a better understanding of their knowledge. So, in order to overcome this hurdle, we utilised Knowledge Graph Properties and relationships to propagate change in mastery from one concept to another. To give a glance at Knowledge Graph relationships, some of the relationships between concepts are prerequisite, is part of, is an example, and other granular relationships. In short, we keep track of a student’s mastery during his time on the platform at the concept level and also propagate the mastery to other related concepts to handle data sparsity.

Now, talking about the actual implementation, as with most AI platforms, we started with Bayesian Knowledge Tracing to track student’s mastery of concepts. Although, an easy to implement, widely popular and one of the very first knowledge tracing models, it had its limitations such as it cannot support multi-skill activities, it considers all concepts independent and hence, it isn’t able to capture the sequential journey and impact of one concept on other. To overcome this, we then implemented Vanilla Deep Knowledge Tracing which allowed us a greater degree of freedom and enabled us to capture the impact of one concept on another and propagate mastery accordingly to all related concepts. But Vanilla Deep Knowledge Tracing being a deep learning model requires a lot of data and generally interprets signal of statistical value. So sometimes, even though a student performs a question correctly or incorrectly, for a few concepts mastery decreased or increased respectively. Another problem with Vanilla Deep Knowledge Tracing is the interpretability. It does not provide an explanation about the amount of mastery update, amount of mastery propagation or how it decides the concepts to which mastery needs to be propagated. Digging for causal, we identified it was the lack of sufficient data on all concepts. Therefore, we looked for methods that allowed us to mitigate data insufficiency. We utilised the existing knowledge graph and were able to improve the model performance without the need for a surplus amount of data. Another byproduct was the interpretability of updates across concepts which a Vanilla Deep Knowledge Tracing model was not able to provide.

In conclusion, Concept Mastery is the task of modelling and finding the current mastery of student on the concept. At Embibe, Concept Mastery is one of the pillars of the Embibe Score Quotient that keep tracks of student’s Academic Quotient in order to recommend content required to fill the student’s learning gap. It keeps track of student’s mastery across all concepts of the knowledge graph and it takes into account the overlapping nature of concepts through mastery propagation. In the future, we will explore ways to infuse domain knowledge to enrich Concept Mastery for better model performance without additional data or losing interpretability.

References

[1] Mastery learning