Skill Them Early

There is a familiar meme that takes a light-hearted view on the current state of many education systems around the world. Judging a fish by its ability to climb a tree is counter-intuitive at best and sadistic at worst.

Figure 1: The “Unfairness” of a “Fair” Education System

And yet, this is what many parents, teachers, and education systems expect from students. A common complaint of formal education is that it’s very hard to identify the students’ inherent skills and then invest the appropriate set of resources to nurture and nourish those skills.

At Embibe, we beg to differ. Leveraging advances in content and user modelling and mining the extensive interaction data collected as students use our platform, we can quantitatively measure student skill levels across a range of concepts. Here is a peek at the volume of data that Embibe has access to be able to work on this problem:

• Over 75+ million total sessions and 5.5+ million hours of time spent over 5+ years,
• Over 90 million attempts, with over 24 billion metadata associated with insights mining,
• Over 40K concepts with more than 700K interconnections on Embibe’s Knowledge Graph,
• Billions of clickstream events totalling terabytes of academic data for insight mining.

Let us take a look at skill proficiency for various flavours of exams (streams). For our analysis, we looked at:

• JEE (which focuses on Physics, Chemistry, and Mathematics),
• NEET (which focuses on Physics, Chemistry, and Biology),
• and K12 (which includes Physics, Chemistry, Mathematics, and Biology).

Skill Calibration

Embibe’s Data Science Lab has mined millions of attempts by students, in the highest-performing cohort, on questions that have been tagged to 9 skill types required to solve them. These skill types are Analytical, Calculation, Deductive, Intuitive, Manipulations, Memory, Verbal Comprehension, Visualization, and Abstract. Questions can be tagged to one or more skills. Embibe has developed a Smart Tagging system that uses a combination of human tagging using expert faculty and NLP-based automated tagging to assign tags to questions.

Figure 2 shows an animated radar plot tracking students’ relative scaled average skill proficiency on K12, JEE, and NEET-related questions.

Figure 2: An Animated Radar Plot Showing the Relative Scaled Average Skill Proficiency of Students on K12, JEE, NEET Related Questions. 

Insights Derived

As can be seen in the plot, there is a clear divide in skill proficiency between questions tagged to different streams. This difference is most apparent between JEE and NEET.

• JEE requires high proficiency levels for the skills of Calculation, Analytical, Manipulation, Deductive, and Memory.
• NEET requires high proficiency in Visualisation, Abstract, and Memory. In fact, very high proficiency is required in the calculation skill for JEE compared to NEET. This corresponds to the fact that JEE tests Mathematics while NEET does not.
• Also, Visualisation requires very high proficiency for NEET compared to JEE because NEET tests Biology, which has many diagrams and drawings, while JEE does not.
• The K12 stream, however, requires moderate proficiency levels for most skills barring Memory. Memory is one skill that is quite important regardless of the stream. Recall that questions can be tagged with multiple skills needed to solve them. And for all streams, Memory is required for various functions like recalling knowledge, formulae, equations, reactions, diagrams, etc.

Potential Applications

Using empirical insights on skill proficiency by mining rich student attempt data makes guiding students according to their inherent skills possible. For instance, by tracking the skills at which a particular student is proficient early on, we can suggest that the student focuses on one stream over another. Students who have an inherent proficiency in the skills of Analytical and Calculation, for instance, can be encouraged to focus on Mathematics and hence, the JEE exam as opposed to students with inherent proficiency in the skill of Visualisation, who can be encouraged to focus on Biology and hence, the NEET exam.

By using data-driven insights on skill proficiency right from lower grades, we can save billions of person-years each generation as we skill students by guiding them to leverage their inherent skills, thereby effectively preparing them for the job market.

Embibe has always believed that the power of data unleashed by advanced algorithms can help improve student learning outcomes at scale. Identifying a student’s inherent skills early on, thereby enabling timely intervention and guidance, is another step on that journey.