{"id":127,"date":"2021-11-29T17:18:18","date_gmt":"2021-11-29T17:18:18","guid":{"rendered":"http:\/\/192.168.64.4\/in-en\/?post_type=ai&#038;p=127"},"modified":"2023-06-12T08:05:56","modified_gmt":"2023-06-12T08:05:56","slug":"intelligent-content-ingestion","status":"publish","type":"ai","link":"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/intelligent-content-ingestion\/","title":{"rendered":"Intelligent Content Ingestion"},"content":{"rendered":"<p><span style=\"font-weight: 400;\">When we say students can solve unlimited questions on Embibe\u2019s platform, we mean it. Embibe has a large dataset of questions available for students to practice on or to take assessment tests. However, ingesting these questions into the system is tedious and time-consuming.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Historically, this dataset of questions was prepared by human data entry operators who would source questions from various freely available question sets on the internet or via tie-ups with our partner institutions. They would manually type out these questions into a web-based UI. The data is then stored in Embibe\u2019s database and made available to other downstream modules before serving students.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">The question input process that the data entry operators would follow typically involved multiple manually intensive steps:<\/span><\/p>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Identifying the question types,\u00a0<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Typing out the question body information,\u00a0<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Formatting mathematical and scientific symbols and notations,\u00a0<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Identifying images and diagrams in questions,\u00a0<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Extracting the images at the correct resolution,\u00a0<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Marking their locations in the questions with image tags,\u00a0<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Listing out the answer options for single and multiple-answer choice question types,\u00a0<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Recording the correct answer choice if provided, and\u00a0<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Recording miscellaneous information like answer explanations, hints, and\/or tips to solve the question if present in the source document.\u00a0<\/span><\/li>\n<\/ul>\n<p><span style=\"font-weight: 400;\">This process is ripe for automation, allowing easy scaling up of content ingestion, thereby reducing costs. As we expand our content to thousands of exams over hundreds of syllabi, automating content and question ingestion has become the need of the hour.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Embibe has developed an in-house ingestion system \u2014 a hybrid parser framework that can rapidly adapt to multiple input templates. All modules of the framework are self-contained, which allows for easy enhancements and replacements. When a new template style needs to be ingested, independent developers can rapidly update each module by updating the text pattern matches and parser\/extractor scripts.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">While this particular step is a manual process, it allows us to achieve an acceptable compromise between content coverage to be ingested vs the speed of rolling out solutions for new ingestion templates. <\/span><\/p>\n<p><img decoding=\"async\" class=\"aligncenter size-full wp-image-481957\" src=\"https:\/\/exams-assets.embibe.com\/exams\/wp-content\/uploads\/2021\/11\/03232149\/ai181.png\" alt=\"\" width=\"512\" height=\"294\" \/><\/p>\n<p><b><i>Figure 1: Schematic Overview of Embibe\u2019s Hybrid Parser Framework for Automated Content Ingestion<\/i><\/b><\/p>\n<p>We measured the framework\u2019s performance recall and precision, as shown in Table 1 below. We can determine a trade-off on acceptable precision rates by controlling for the desired recall rate.<\/p>\n<p><img decoding=\"async\" class=\"aligncenter size-full wp-image-481958\" src=\"https:\/\/exams-assets.embibe.com\/exams\/wp-content\/uploads\/2021\/11\/03232151\/ai191.png\" alt=\"\" width=\"280\" height=\"158\" \/><\/p>\n<p><b><i>Table 1: Performance of Automated Ingestion on Test Data<\/i><\/b><\/p>\n","protected":false},"featured_media":0,"template":"","yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v21.1 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>AI in Education: Automating Content Ingestion with Embibe&#039;s Intelligent System<\/title>\n<meta name=\"description\" content=\"Embibe&#039;s hybrid parser framework - Automate question input process for scalable and cost-effective content management on their AI-powered learning platform.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/intelligent-content-ingestion\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"AI in Education: Automating Content Ingestion with Embibe&#039;s Intelligent System\" \/>\n<meta property=\"og:description\" content=\"Embibe&#039;s hybrid parser framework - Automate question input process for scalable and cost-effective content management on their AI-powered learning platform.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/intelligent-content-ingestion\/\" \/>\n<meta property=\"og:site_name\" content=\"EMBIBE - The most powerful AI-powered learning platform\" \/>\n<meta property=\"article:modified_time\" content=\"2023-06-12T08:05:56+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/exams-assets.embibe.com\/exams\/wp-content\/uploads\/2021\/11\/03232149\/ai181.png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/intelligent-content-ingestion\/\",\"url\":\"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/intelligent-content-ingestion\/\",\"name\":\"AI in Education: Automating Content Ingestion with Embibe's Intelligent System\",\"isPartOf\":{\"@id\":\"https:\/\/www.embibe.com\/in-en\/#website\"},\"datePublished\":\"2021-11-29T17:18:18+00:00\",\"dateModified\":\"2023-06-12T08:05:56+00:00\",\"description\":\"Embibe's hybrid parser framework - Automate question input process for scalable and cost-effective content management on their AI-powered learning platform.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/intelligent-content-ingestion\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/intelligent-content-ingestion\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/intelligent-content-ingestion\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.embibe.com\/in-en\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Intelligent Content Ingestion\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.embibe.com\/in-en\/#website\",\"url\":\"https:\/\/www.embibe.com\/in-en\/\",\"name\":\"EMBIBE - The most powerful AI-powered learning platform\",\"description\":\"Just another WordPress site\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.embibe.com\/in-en\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"AI in Education: Automating Content Ingestion with Embibe's Intelligent System","description":"Embibe's hybrid parser framework - Automate question input process for scalable and cost-effective content management on their AI-powered learning platform.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/intelligent-content-ingestion\/","og_locale":"en_US","og_type":"article","og_title":"AI in Education: Automating Content Ingestion with Embibe's Intelligent System","og_description":"Embibe's hybrid parser framework - Automate question input process for scalable and cost-effective content management on their AI-powered learning platform.","og_url":"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/intelligent-content-ingestion\/","og_site_name":"EMBIBE - The most powerful AI-powered learning platform","article_modified_time":"2023-06-12T08:05:56+00:00","og_image":[{"url":"https:\/\/exams-assets.embibe.com\/exams\/wp-content\/uploads\/2021\/11\/03232149\/ai181.png"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/intelligent-content-ingestion\/","url":"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/intelligent-content-ingestion\/","name":"AI in Education: Automating Content Ingestion with Embibe's Intelligent System","isPartOf":{"@id":"https:\/\/www.embibe.com\/in-en\/#website"},"datePublished":"2021-11-29T17:18:18+00:00","dateModified":"2023-06-12T08:05:56+00:00","description":"Embibe's hybrid parser framework - Automate question input process for scalable and cost-effective content management on their AI-powered learning platform.","breadcrumb":{"@id":"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/intelligent-content-ingestion\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/intelligent-content-ingestion\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/intelligent-content-ingestion\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.embibe.com\/in-en\/"},{"@type":"ListItem","position":2,"name":"Intelligent Content Ingestion"}]},{"@type":"WebSite","@id":"https:\/\/www.embibe.com\/in-en\/#website","url":"https:\/\/www.embibe.com\/in-en\/","name":"EMBIBE - The most powerful AI-powered learning platform","description":"Just another WordPress site","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.embibe.com\/in-en\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/www.embibe.com\/in-en\/wp-json\/wp\/v2\/ai\/127"}],"collection":[{"href":"https:\/\/www.embibe.com\/in-en\/wp-json\/wp\/v2\/ai"}],"about":[{"href":"https:\/\/www.embibe.com\/in-en\/wp-json\/wp\/v2\/types\/ai"}],"wp:attachment":[{"href":"https:\/\/www.embibe.com\/in-en\/wp-json\/wp\/v2\/media?parent=127"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}