{"id":120,"date":"2021-11-29T17:13:49","date_gmt":"2021-11-29T17:13:49","guid":{"rendered":"http:\/\/192.168.64.4\/in-en\/?post_type=ai&#038;p=120"},"modified":"2023-06-12T09:38:43","modified_gmt":"2023-06-12T09:38:43","slug":"data-is-the-new-force","status":"publish","type":"ai","link":"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/data-is-the-new-force\/","title":{"rendered":"Data Is the New Force"},"content":{"rendered":"<p>Embibe is fanatical about instrumenting, measuring, collecting, mining, and archiving data. Embibe owns its data, our IP depends on it. At Embibe, we delay the release until adequate instrumentation is in place to measure how our users interact with our products and what factors lead to specific outcomes. This obsession with data led to many insightful revelations on how students study and achieve their goals.<\/p>\n<p>For instance, a student\u2019s potential to score is a combination of two factors \u2013 their learning ability which contributes ~61% of the overall potential to score, and their behavioural attributes, which contribute ~39%. This razor-sharp focus on being data-driven has enabled Embibe to build products that personalise education and deliver tremendous improvements in student learning outcomes.<\/p>\n<h3><b>Primary Data Collection<\/b><\/h3>\n<p><span style=\"font-weight: 400;\">Data is instrumented and collected at various stages and locations all across Embibe\u2019s platform. It is not just necessary to capture data as much as capturing the correct type of data, at the right time, in the proper context, and with the right level of granularity. Data capture at Embibe falls broadly into the following categories:<\/span><\/p>\n<ul>\n<li aria-level=\"1\"><b>Instrumentation of Rich Event Types:<\/b>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><span style=\"font-weight: 400;\">User-interaction explicit events \u2013 clicks, taps, hovers, scrolls, text updates,<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><span style=\"font-weight: 400;\">User-interaction implicit events \u2013 cursor position, tap pressure, device orientation, location,<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><span style=\"font-weight: 400;\">System-generated server-side events \u2013 page load, session refreshes, API calls,<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><span style=\"font-weight: 400;\">System-generated client-side events \u2013 system push notifications and triggers.<\/span><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<ul>\n<li aria-level=\"1\"><b>Specific Data by Property:<\/b><\/li>\n<\/ul>\n<ul>\n<li style=\"list-style-type: none;\">\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><span style=\"font-weight: 400;\">Page views (URL, referrer, user agent, device, IP, timestamp, traffic source, campaign),<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><span style=\"font-weight: 400;\">Practice attempt level data (timestamp, visit\/re-visit, answer choice, time first seen, correct, time spent, solution viewed, hint used) \u2013 aggregated at the session level,<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><span style=\"font-weight: 400;\">Learn behaviour data:<\/span>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"3\"><span style=\"font-weight: 400;\">Search event data (timestamp, query, result set),<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"3\"><span style=\"font-weight: 400;\">Result interaction data (timestamp, suggested result selected, result widget and context, widget position),<\/span><\/li>\n<\/ul>\n<\/li>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><span style=\"font-weight: 400;\">Test attempt event-level data (timestamp, visit\/re-visit, answer choice, time first seen, correct, time spent, feedback viewed) \u2013 aggregated at the session level,<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><span style=\"font-weight: 400;\">Doubt Resolutions (Academic Forum) question and answer details, timestamps, and user voting behaviour.<\/span><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p><span style=\"font-weight: 400;\">Many practical considerations need to be accounted for when instrumenting the data collection at the scale that Embibe does. For instance, we rely on several modalities to gather all this data. The user interaction event stream is logged by integrating with third-party plugins<\/span><span style=\"font-weight: 400;\">. Server-side page load and session event logging is instrumented in-house and pushed to NoSQL databases. Daily user activity data on properties like Practice and Test are stored in the DB for query aggregations by the front end.<\/span><\/p>\n<h3><b>Data Processing<\/b><\/h3>\n<p><span style=\"font-weight: 400;\">Once primary data collection happens, it is necessary to clean, enrich, mine, and visualise it. At Embibe, we have the following broad approaches to using the data we collect:<\/span><\/p>\n<ul>\n<li aria-level=\"1\"><b>In-house Reporting and Ad-hoc Analysis:<\/b><\/li>\n<\/ul>\n<ul>\n<li style=\"list-style-type: none;\">\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><span style=\"font-weight: 400;\">Log mining to generate reporting data for traffic patterns, user monetisation, test-on-test improvement, search failures, and other needs. The processed data is again pushed into Elasticsearch and visualised using the Kibana dashboard.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><span style=\"font-weight: 400;\">Primary raw data is stored on HBase over HDFS for any required analysis to be conducted on an ad-hoc basis.<\/span><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<h5><img decoding=\"async\" class=\"aligncenter size-full wp-image-481878\" src=\"https:\/\/exams-assets.embibe.com\/exams\/wp-content\/uploads\/2021\/11\/03222312\/ai101.png\" alt=\"\" width=\"512\" height=\"434\" \/><\/h5>\n<p><em>Figure 1: A High-Level Schematic of the Data Flow Stack that Powers the Intelligence that Embibe\u2019s Data Science Lab Develops<\/em><\/p>\n<ul>\n<li aria-level=\"1\"><b>Third-Party Tools for Business\/Product\/Marketing Self-Serve<\/b><\/li>\n<\/ul>\n<ul>\n<li style=\"list-style-type: none;\">\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><span style=\"font-weight: 400;\">Google Analytics for broad-level traffic monitoring, including traffic sources, demographic and location information, device breakdown, page views, time spent, and retention metrics.<\/span><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n","protected":false},"featured_media":0,"template":"","yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v21.1 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Data is the New Force: Embibe&#039;s Data-Driven Approach to Personalised Education<\/title>\n<meta name=\"description\" content=\"Discover how Embibe leverages data to drive insights and improvements in learning outcomes. Explore how Embibe&#039;s data-driven approach is shaping the future of education.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/data-is-the-new-force\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Data is the New Force: Embibe&#039;s Data-Driven Approach to Personalised Education\" \/>\n<meta property=\"og:description\" content=\"Discover how Embibe leverages data to drive insights and improvements in learning outcomes. Explore how Embibe&#039;s data-driven approach is shaping the future of education.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/data-is-the-new-force\/\" \/>\n<meta property=\"og:site_name\" content=\"EMBIBE - The most powerful AI-powered learning platform\" \/>\n<meta property=\"article:modified_time\" content=\"2023-06-12T09:38:43+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/exams-assets.embibe.com\/exams\/wp-content\/uploads\/2021\/11\/03222312\/ai101.png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/data-is-the-new-force\/\",\"url\":\"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/data-is-the-new-force\/\",\"name\":\"Data is the New Force: Embibe's Data-Driven Approach to Personalised Education\",\"isPartOf\":{\"@id\":\"https:\/\/www.embibe.com\/in-en\/#website\"},\"datePublished\":\"2021-11-29T17:13:49+00:00\",\"dateModified\":\"2023-06-12T09:38:43+00:00\",\"description\":\"Discover how Embibe leverages data to drive insights and improvements in learning outcomes. Explore how Embibe's data-driven approach is shaping the future of education.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/data-is-the-new-force\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/data-is-the-new-force\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/data-is-the-new-force\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.embibe.com\/in-en\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Data Is the New Force\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.embibe.com\/in-en\/#website\",\"url\":\"https:\/\/www.embibe.com\/in-en\/\",\"name\":\"EMBIBE - The most powerful AI-powered learning platform\",\"description\":\"Just another WordPress site\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.embibe.com\/in-en\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Data is the New Force: Embibe's Data-Driven Approach to Personalised Education","description":"Discover how Embibe leverages data to drive insights and improvements in learning outcomes. Explore how Embibe's data-driven approach is shaping the future of education.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/data-is-the-new-force\/","og_locale":"en_US","og_type":"article","og_title":"Data is the New Force: Embibe's Data-Driven Approach to Personalised Education","og_description":"Discover how Embibe leverages data to drive insights and improvements in learning outcomes. Explore how Embibe's data-driven approach is shaping the future of education.","og_url":"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/data-is-the-new-force\/","og_site_name":"EMBIBE - The most powerful AI-powered learning platform","article_modified_time":"2023-06-12T09:38:43+00:00","og_image":[{"url":"https:\/\/exams-assets.embibe.com\/exams\/wp-content\/uploads\/2021\/11\/03222312\/ai101.png"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/data-is-the-new-force\/","url":"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/data-is-the-new-force\/","name":"Data is the New Force: Embibe's Data-Driven Approach to Personalised Education","isPartOf":{"@id":"https:\/\/www.embibe.com\/in-en\/#website"},"datePublished":"2021-11-29T17:13:49+00:00","dateModified":"2023-06-12T09:38:43+00:00","description":"Discover how Embibe leverages data to drive insights and improvements in learning outcomes. Explore how Embibe's data-driven approach is shaping the future of education.","breadcrumb":{"@id":"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/data-is-the-new-force\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/data-is-the-new-force\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.embibe.com\/in-en\/artificial-intelligence-ai-in-education\/data-is-the-new-force\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.embibe.com\/in-en\/"},{"@type":"ListItem","position":2,"name":"Data Is the New Force"}]},{"@type":"WebSite","@id":"https:\/\/www.embibe.com\/in-en\/#website","url":"https:\/\/www.embibe.com\/in-en\/","name":"EMBIBE - The most powerful AI-powered learning platform","description":"Just another WordPress site","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.embibe.com\/in-en\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/www.embibe.com\/in-en\/wp-json\/wp\/v2\/ai\/120"}],"collection":[{"href":"https:\/\/www.embibe.com\/in-en\/wp-json\/wp\/v2\/ai"}],"about":[{"href":"https:\/\/www.embibe.com\/in-en\/wp-json\/wp\/v2\/types\/ai"}],"wp:attachment":[{"href":"https:\/\/www.embibe.com\/in-en\/wp-json\/wp\/v2\/media?parent=120"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}