{"id":166,"date":"2021-11-29T19:34:04","date_gmt":"2021-11-29T19:34:04","guid":{"rendered":"http:\/\/192.168.64.4\/in-en\/?post_type=ai&p=166"},"modified":"2023-06-12T07:25:35","modified_gmt":"2023-06-12T07:25:35","slug":"automated-evaluation-of-free-text-answer-based-questions","status":"publish","type":"ai","link":"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/automated-evaluation-of-free-text-answer-based-questions\/","title":{"rendered":"Automated Evaluation of Free Text Answer based Questions"},"content":{"rendered":"\n

A vast majority of competitive exams require the participants to solve objective-type questions, like questions that require one or more correct answers to be selected from a given set of answer choices or questions for which participants have to enter a numerical value. Evaluation of tests that are based on objective-type questions is quite straightforward.<\/p>\n\n\n\n

However, there are many exams, for example, board exams, which include questions with free-text answers. Evaluation of free-text answers is still an open research problem with some successful solutions that target essay scoring. Developing a generic evaluator that can score free text answers of various styles across different academic domains requires using advanced NLP\/NLU. It is an area of interest to Embibe.<\/p>\n\n\n\n

We can divide the problem into two sub-problems.<\/p>\n\n\n\n

    \n
  1. Entity linking<\/li>\n\n\n\n
  2. Semantic similarity<\/li>\n<\/ol>\n\n\n\n

    Entity linking<\/strong><\/h3>\n\n\n\n

    In entity linking, we can include short forms\/acronyms and aka(also known as) type entities. For example, acronyms like<\/p>\n\n\n\n

    “PMC “: “pollen mother cell”,<\/p>\n\n\n\n

    “MMC “: “megaspore mother cell”,<\/p>\n\n\n\n

    “PEN”: “primary endosperm nucleus”,<\/p>\n\n\n\n

    “PEC”: “primary endosperm cell”,<\/p>\n\n\n\n

    “LH “: “luteinizing hormone”,<\/p>\n\n\n\n

    “FSH “: “follicle stimulating hormone”<\/p>\n\n\n\n

    And aka like,<\/p>\n\n\n\n

    “mushroom”: “toadstool”,<\/p>\n\n\n\n

    “germs”: “microbes”,<\/p>\n\n\n\n

    “bacteria”: “microbes”,<\/p>\n\n\n\n

    “yeast”: “microbes”,<\/p>\n\n\n\n

    “renewable”: “inexhaustible”,<\/p>\n\n\n\n

    “traits”: “characteristics”,<\/p>\n\n\n\n

    We can also map chemical name entities like,<\/p>\n\n\n\n

    ‘(NH4)(NO3)’: ‘Ammonium nitrate’,<\/p>\n\n\n\n

    ‘(NH4)2C2O4’: ‘Ammonium oxalate’,<\/p>\n\n\n\n

    ‘Ag2O’: ‘Silver oxide’,<\/p>\n\n\n\n

    ‘Ag2SO4’: ‘Silver sulfate’,<\/p>\n\n\n\n

    ‘Al(NO3)3’: ‘Aluminium nitrate’,<\/p>\n\n\n\n

    We can map acronyms and similar words using these mappings and then match them with actual answers.<\/p>\n\n\n\n

    Semantic Similarity<\/h3>\n\n\n\n

    Two sentences could mean the same thing. We establish semantic similarity using domain-infused knowledge and inferring language model probabilities.<\/p>\n\n\n\n

    We can get the embedding of the student\u2019s answer and compare it with the embedding of the actual answer. If the cosine distance between them is lower than a certain threshold, then we can consider them similar and mark the answer as correct.<\/p>\n\n\n\n

    We can use self-attention-based models like BERT[1] and RoBERTa[2] to get the embeddings of the student\u2019s answer and correct answer, and calculate the cosine distance between them to get their similarity.<\/p>\n\n\n\n

    References:<\/strong><\/p>\n\n\n\n

    [1] Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova. \u201cBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding\u201d<\/a><\/p>\n\n\n\n

    [2] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, Veselin Stoyanov. \u201cRoBERTa: A Robustly Optimized BERT Pretraining Approach\u201d<\/a><\/p>\n","protected":false},"featured_media":0,"template":"","yoast_head":"\nAutomated Evaluation of Free Text Answers in AI Education - Embibe<\/title>\n<meta name=\"description\" content=\"Explore how Embibe utilizes advanced NLP\/NLU techniques to solve the challenge of evaluating free text answers in academic exams.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/automated-evaluation-of-free-text-answer-based-questions\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Automated Evaluation of Free Text Answers in AI Education - Embibe\" \/>\n<meta property=\"og:description\" content=\"Explore how Embibe utilizes advanced NLP\/NLU techniques to solve the challenge of evaluating free text answers in academic exams.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/automated-evaluation-of-free-text-answer-based-questions\/\" \/>\n<meta property=\"og:site_name\" content=\"EMBIBE - The most powerful AI-powered learning platform\" \/>\n<meta property=\"article:modified_time\" content=\"2023-06-12T07:25:35+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/automated-evaluation-of-free-text-answer-based-questions\/\",\"url\":\"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/automated-evaluation-of-free-text-answer-based-questions\/\",\"name\":\"Automated Evaluation of Free Text Answers in AI Education - Embibe\",\"isPartOf\":{\"@id\":\"https:\/\/www.embibe.com\/in-en\/#website\"},\"datePublished\":\"2021-11-29T19:34:04+00:00\",\"dateModified\":\"2023-06-12T07:25:35+00:00\",\"description\":\"Explore how Embibe utilizes advanced NLP\/NLU techniques to solve the challenge of evaluating free text answers in academic exams.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/automated-evaluation-of-free-text-answer-based-questions\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/automated-evaluation-of-free-text-answer-based-questions\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/automated-evaluation-of-free-text-answer-based-questions\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.embibe.com\/in-en\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Automated Evaluation of Free Text Answer based Questions\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.embibe.com\/in-en\/#website\",\"url\":\"https:\/\/www.embibe.com\/in-en\/\",\"name\":\"EMBIBE - The most powerful AI-powered learning platform\",\"description\":\"Just another WordPress site\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.embibe.com\/in-en\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Automated Evaluation of Free Text Answers in AI Education - Embibe","description":"Explore how Embibe utilizes advanced NLP\/NLU techniques to solve the challenge of evaluating free text answers in academic exams.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/automated-evaluation-of-free-text-answer-based-questions\/","og_locale":"en_US","og_type":"article","og_title":"Automated Evaluation of Free Text Answers in AI Education - Embibe","og_description":"Explore how Embibe utilizes advanced NLP\/NLU techniques to solve the challenge of evaluating free text answers in academic exams.","og_url":"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/automated-evaluation-of-free-text-answer-based-questions\/","og_site_name":"EMBIBE - The most powerful AI-powered learning platform","article_modified_time":"2023-06-12T07:25:35+00:00","twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/automated-evaluation-of-free-text-answer-based-questions\/","url":"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/automated-evaluation-of-free-text-answer-based-questions\/","name":"Automated Evaluation of Free Text Answers in AI Education - Embibe","isPartOf":{"@id":"https:\/\/www.embibe.com\/in-en\/#website"},"datePublished":"2021-11-29T19:34:04+00:00","dateModified":"2023-06-12T07:25:35+00:00","description":"Explore how Embibe utilizes advanced NLP\/NLU techniques to solve the challenge of evaluating free text answers in academic exams.","breadcrumb":{"@id":"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/automated-evaluation-of-free-text-answer-based-questions\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/automated-evaluation-of-free-text-answer-based-questions\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/automated-evaluation-of-free-text-answer-based-questions\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.embibe.com\/in-en\/"},{"@type":"ListItem","position":2,"name":"Automated Evaluation of Free Text Answer based Questions"}]},{"@type":"WebSite","@id":"https:\/\/www.embibe.com\/in-en\/#website","url":"https:\/\/www.embibe.com\/in-en\/","name":"EMBIBE - The most powerful AI-powered learning platform","description":"Just another WordPress site","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.embibe.com\/in-en\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/www.embibe.com\/in-en\/wp-json\/wp\/v2\/ai\/166"}],"collection":[{"href":"https:\/\/www.embibe.com\/in-en\/wp-json\/wp\/v2\/ai"}],"about":[{"href":"https:\/\/www.embibe.com\/in-en\/wp-json\/wp\/v2\/types\/ai"}],"wp:attachment":[{"href":"https:\/\/www.embibe.com\/in-en\/wp-json\/wp\/v2\/media?parent=166"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}