This study investigates the possible use of AI-driven chatbots, such as ChatGPT, to assist in the process of cybersecurity education, particularly in the field of threat hunting. Given the course's sensitive and secure nature, it remains questionable if ChatGPT or similar systems would possess the capability to assist human hunters. An experiment was conducted with 35 students who had just finished the course independently, without utilizing ChatGPT. The participants were tasked with assessing the ability of ChatGPT to provide correct responses to crucial questions pertaining to the course and assist them in actual threat-hunting scenarios. The data was collected from 38 questions formulated for ChatGPT and 3 supplementary questions for students. The results show that 57.8% of students believed that ChatGPT's answers accurately represented their knowledge on the topic. Additionally, 20.6% of students discovered new information through ChatGPT's responses, while 12.1% found the answers to be insufficient and 9.6% considered them irrelevant. Regarding further inquiries, students have identified the primary advantage of using ChatGPT as comprehensive knowledge coverage, while the main disadvantage is its inability to disclose timely and sensitive information. The third supplementary inquiry evaluated whether students were still obligated to physically attend class if they had access to ChatGPT. They confirmed their attendance, acknowledging that they still needed to go to class as the lecturer taught them how to utilize various hunting tools to enhance their skills in threat hunting. They were also informed that they would learn how to effectively inquire using ChatGPT at a later time.

Authors:
Muhammad Gilvy Langgawan Putra, National Taiwan University of Science and Technology, Taiwan
Raymund Lin, National Taiwan University of Science and Technology, Taiwan

About the Presenter(s)
Gilvy Langgawan is a Ph.D. student at the National Taiwan University of Science and Technology (Taiwan Tech). Currently, my research focuses on the Large Language Model (LLM) in cybersecurity education, and I am now developing a platform for that.

Connect on Linkedin
https://www.linkedin.com/in/muhammad-gilvy-langgawan-putra-65042a12b/

Connect on ResearchGate
https://www.researchgate.net/profile/Gilvy-Langgawan

See this presentation on the full schedule – Sunday Schedule

Bookmark