![\"\"](\"https:\/\/assets.embibe.com\/production\/web-assets\/images\/AI\/articles\/medhas_img1.png\" "\\"\\"\/")
<\/figure>\n\n\n\nWe infuse knowledge embedding from knowledge graphs into self-attention-based models. For example, we took the embedding of conceptual entities like \u201cabsent-mindedness\u201d from ConceptNet and infuse it with token embeddings of the same into BERT. The same process of infusing knowledge embedding for \u201cabsence of mind\u201d. So, the next self-attention layer will pay attention to the combined knowledge and token embeddings of \u201cabsent-mindedness\u201d and \u201cabsence of mind\u201d entities.<\/p>\n\n\n\n
As knowledge embedding of \u201cabsent-mindedness\u201d and \u201cabsence of mind\u201d will be very similar, the model will get the idea that these two are similar entities.<\/p>\n\n\n\n In the above example, we fetch knowledge embedding of ambiguous entities like \u201cmass\u201d and \u201cradius\u201d from WordNet and infuse it with token embeddings of the same into the self-attention layer of BERT.<\/p>\n\n\n\n So, the next self-attention layer will pay attention to combined knowledge embedding and token embedding of the ambiguous entities, which will help the model understand the input sentence better.<\/p>\n\n\n\n How infusing knowledge gives more interpretable confidence:<\/p>\n\n\n\n We can see in the above example(second last row) that, after adding Conceptual entities like \u201cGreenhouse Effect\u201d and ambiguous entities like \u201cTrapping\u201d and \u201cAbsorption\u201d, not only did it make the correct prediction, but the confidence of making the prediction also increased to 60.55% as we added more entities.<\/p>\n\n\n\n When we also added more ambiguous and conceptual entities like, \u201cSolar Energy\u201d and \u201cCause\u201d, respectively, confidence increased even more to 61.97%.<\/p>\n\n\n\n To summarise, as we add more meaningful entities, not only it made correct predictions(unlike vanilla BERT was making wrong predictions), but it also increased the confidence of the model.<\/p>\n\n\n\n Platform Medhas is built by the Education Research Lab at Embibe. Education Research Lab is a niche disciplined effort to identify key open problems with colossal impact and push state-of-the-art research by inventing original value propositions.<\/p>\n\n\n\n [1] Gaur, Manas, Keyur Faldu, and Amit Sheth. “Semantics of the Black-Box: Can knowledge graphs help make deep learning systems more interpretable and explainable?.” <\/a>IEEE Internet Computing<\/a> 25, no. 1 (2021): 51-59.<\/a><\/p>\n\n\n\n [2] Faldu, Keyur, Amit Sheth, Prashant Kikani, and Hemang Akabari. “KI-BERT: Infusing Knowledge Context for Better Language and Domain Understanding.” <\/a>arXiv preprint arXiv:2104.08145<\/a> (2021).<\/a><\/p>\n\n\n\n [3] <\/a>Gaur, Manas, Ankit Desai, Keyur Faldu, and Amit Sheth. “Explainable AI Using Knowledge Graphs.” In <\/a>ACM CoDS-COMAD Conference<\/a>. 2020.<\/a><\/p>\n\n\n\n [4] Keyur Faldu. \u201cRise of Modern NLP and the Need of Interpretability!<\/a>\u201d Towards Data Science, August 2020.<\/p>\n\n\n\n [5] Keyur Faldu, Dr Amit Sheth. \u201cDiscovering the Encoded Linguistic Knowledge in NLP models<\/a>.\u201d Towards Data Science, September 2020.<\/p>\n\n\n\n [6] Keyur Faldu, Dr Amit Sheth. \u201cLinguistics Wisdom of NLP Models<\/a>.\u201d Towards Data Science, November 2020.<\/p>\n\n\n\n [7] A. Sheth, M. Gaur, K. Roy and K. Faldu, “Knowledge-Intensive Language Understanding for Explainable AI,” in IEEE Internet Computing, vol. 25, no. 5, pp. 19-24, 1 Sept.-Oct. 2021, doi: 10.1109\/MIC.2021.3101919<\/a><\/p>\n\n\n\n [8] Faldu, Keyur, Aditi Avasthi, and Achint Thomas. “Adaptive learning machine for score improvement and parts thereof.” U.S. Patent 10,854,099, issued December 1, 2020.<\/p>\n\n\n\n [9] Dhavala, Soma, Chirag Bhatia, Joy Bose, Keyur Faldu, and Aditi Avasthi. “Auto Generation of Diagnostic Assessments and Their Quality Evaluation.” International Educational Data Mining Society (2020).<\/p>\n\n\n\n [10] Faldu, Keyur, Achint Thomas, and Aditi Avasthi. “System and method for recommending personalized content using contextualized knowledge base.” U.S. Patent Application 16\/586,512, filed October 1, 2020.<\/p>\n\n\n\n [11] Thomas, Achint, Keyur Faldu, and Aditi Avasthi. “System and method for personalized retrieval of academic content in a hierarchical manner.” U.S. Patent Application 16\/740,223, filed October 1, 2020.<\/p>\n\n\n\n [12] “#RAISE2020 – Embibe – AI-Powered learning outcomes platform for personalized education”, MyGov India, Oct 2020, <\/a>https:\/\/www.youtube.com\/watch?v=kuwFtHgN3qU<\/a><\/p>\n","protected":false},"featured_media":0,"template":"","yoast_head":"\n![\"\"](\"https:\/\/assets.embibe.com\/production\/web-assets\/images\/AI\/articles\/medhas_img3.png\" "\\"\\"\/")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/assets.embibe.com\/production\/web-assets\/images\/AI\/articles\/medhas_img2.png\" "\\"\\"\/")
<\/figure>\n\n\n\nReference:<\/h5>\n\n\n\n