The logit function, which is the inverse of the logistic function, is used to transform the probability back to a log-odds scale. The logit function is defined as follows:<\/span><\/p>\n Using the 1PL Item Response Theory parameter estimation techniques, we can predict the ability level \u03b8i of a learner, given the data about the learner\u2019s response to each attempted question.<\/span><\/p>\n The 1PL item response theory model is indeed logistic regression with a domain-specific parametrization. Consequently, we can realize such a model using any deep learning framework. The deep learning architecture for the 1PL model is shown in Figure 1.<\/span><\/p>\n Figure 1: Neural Network Architecture for Estimating the 1PL IRT Model Parameters<\/strong><\/em><\/p>\n Our model is implemented in Keras<\/span>[4]<\/span> as a deep neural network. The advantages of modelling the problem as a neural network are:<\/span><\/p>\n We refer to this model as the 1PL Deep Item Response Theory model.<\/span><\/p>\n In order to benchmark and validate the modelling strategy, we generate simulated data as follows:<\/span><\/p>\n We have simulated 100 questions, 100 learners, and one response per learner per question using applications of item response theory to practical testing problems.<\/span><\/p>\n We fit the 1PL Deep Item Response Theory model to the simulated dataset. Inputs to the neural network are the user vector (one-hot encoded) and the question vector (also one-hot encoded), and outputs are the parameters of the Item Response Theory model, including item difficulty, learner ability and prediction of whether the learner will answer correctly or not. The neural network is fully connected. It has two input layers, intermediate layers for difficulty and ability, and one output layer for prediction.<\/span><\/p>\n We compare the 1PL Deep Item Response Theory output from the neural network with the true outputs from the simulated data.<\/span><\/p>\n Model: <\/b>The architecture of 1PL Item Response Theory is defined by harnessing the compositionality of NNs, utilizing Keras functional APIs. The overall model is structured by stacking Dense layers \u2013 here, 2 Dense layers for the 1PL model, each representative of a User or Item parameters pivotal in driving the likelihood (Pij) of a user (i) responding to an Item (j).<\/span><\/p>\n Hyper Parameters: <\/b>The following default settings are used in each dense layer<\/span><\/p>\n A developer can override the above settings, or the best configuration can be obtained by searching over a space of configurations for item response theory parameter estimation techniques. Such details will be in a forthcoming blog. The defined model is flexible enough to extend its usage for two or even three parameters, namely discrimination and guessability, and consequently, in retrospect, an extended model coupled with Neural Architecture Search capability can be constrained to perform as one or two PL models.<\/span><\/p>\n The below plots show the correlation between:<\/span><\/p>\n From the Deep Item Response Theory model trained on 1PL data, the log-likelihood of the 1PL DIRT model is 0.587.<\/span><\/p>\n As we can see, we get a good correlation in all three cases, indicating that our 1PL Deep Item Response Theory model successfully predicts the difficulty, ability and test score with good accuracy.<\/span><\/p>\n Figure 2: Scatterplots of the True Versus Derived Keras Model Difficulty and Ability Parameters<\/em><\/strong><\/p>\n Figure 3: Hexbin Plot of the True Probability of Answering the Questions Correctly Versus. The Probability Derived from our Trained Keras Model<\/em><\/strong><\/p>\n We have shown that, based on simulations, the 1PL Item Response Theory model can be implemented via a Deep Learning model. Using the Item Response Theory parameters, we can get good estimates of learner ability and the difficulty level of the questions using our 1PL Item Response Theory-based model. These estimates can, in turn, be used in generating adaptive tests, goal setting, and other downstream problems.<\/span><\/p>\n References<\/b><\/p>\n\n
A Deep Learning Architecture for 1PL Item Response Theory<\/b><\/h3>\n
![\"\"](\"https:\/\/exams-assets.embibe.com\/exams\/wp-content\/uploads\/2021\/11\/03203420\/ai51.png\")
<\/p>\n\n
Validation<\/b><\/h3>\n
\n
Implementation<\/b><\/h3>\n
\n
Experimental Results<\/b><\/h3>\n
\n
![\"\"](\"https:\/\/exams-assets.embibe.com\/exams\/wp-content\/uploads\/2021\/11\/03203421\/ai61.png\")
![\"\"](\"https:\/\/exams-assets.embibe.com\/exams\/wp-content\/uploads\/2021\/11\/03203424\/ai71.png\")
<\/p>\n![\"\"](\"https:\/\/exams-assets.embibe.com\/exams\/wp-content\/uploads\/2021\/11\/03203426\/ai81.png\")
<\/p>\nConclusion<\/b><\/h3>\n
\n