This study presents the development and evaluation of an AI-powered robotic assistant designed to revolutionize classroom video-based learning. The study focuses on the system's engineering innovations, which leverage advanced natural language processing (NLP) and computer vision techniques to generate interactive multiple-choice questions from educational YouTube videos automatically. The robotic assistant transcribes video content using state-of-the-art speech recognition algorithms and segments it into distinct parts. GPT-based language models generate multiple-choice questions for each segment, projected onto students' desks, creating an immersive and interactive learning experience. The system features a 3D-printed face with a human-like appearance and lip-sync capabilities, enhancing communication and mimicking traditional teaching presence.
Student interaction is facilitated through innovative 'flip-flop' devices with ArUco markers, enabling real-time collection and analysis of responses. This feedback loop allows for dynamic adaptation of video content and questions, tailoring the learning experience to students' needs. The study evaluates the system's performance in a classroom setting, assessing its ability to generate relevant questions, engage students, and adapt to their responses. The findings demonstrate the potential of the AI-powered robotic assistant to enhance video-based learning experiences and reduce teacher workload. This research contributes to the literature on AI-driven educational technologies, highlighting the successful integration of advanced NLP, computer vision, and robotics in transforming video-based learning. The study concludes with future research and development recommendations, focusing on further enhancing the system's capabilities and scalability.

Authors:
Chen Giladi, Sami Shamoon College of Engineering, Israel

About the Presenter(s)
Dr. Chen Giladi, lecturer in the Mechanical Engineering Department at Sami Shamoon College of Engineering (SCE), focuses on AI in education, agricultural & medical robotics and develops advanced machine learning algorithms in these fields.

See this presentation on the full schedule – Friday Schedule

Bookmark