Showing 1009–1022 of 5,042 results for "Abacar Kerem"

Resource 2024 EN

Examining the Influence of Varied Levels of Domain Knowledge Base Inclusion in GPT-based Intelligent Tutors

Blake Castleman · Mehmet Kerem Turkcan

Recent advancements in large language models (LLMs) have facilitated thedevelopment of chatbots with sophisticated conversational capabilities.However, LLMs exhibit frequent inaccurate responses to queries, hinderingapplications in educational settings. In this paper, we investigate theeffectiveness of integrating a knowledge base (KB) with LLM intelligent tutorsto increase response reliability. To achieve this, we design a scaleable KBthat affords educational supervisors seamless integration of lesson curricula,which is automatically processed by the intelligent tutoring system. We thendetail an evaluation, where student participants were presented with questionsabout the artificial intelligence curriculum to respond to. GPT-4 intelligenttutors with varying hierarchies of KB access and human domain experts thenassessed these responses. Lastly, students cross-examined the intelligenttutors' responses to the domain experts' and ranked their various pedagogicalabilities. Results suggest that, although these intelligent tutors stilldemonstrate a lower accuracy compared to domain experts, the accuracy of theintelligent tutors increases when access to a KB is granted. We also observethat the intelligent tutors with KB access exhibit better pedagogical abilitiesto speak like a teacher and understand students than those of domain experts,while their ability to help students remains lagging behind domain experts.

Not Specified