ABSTRACT VIEW
EVALUATING STUDENTS' PERCEPTIONS OF THE EFFECTIVENESS OF A CHAT GPT-DRIVEN PERSONAL ASSISTANT FOR AN INTERCULTURAL COMMUNICATION AND LEADERSHIP COURSE
A. Millward-Sadler
FH Joanneum University of Applied Sciences (AUSTRIA)
The introduction of freely available generative artificial intelligence (GAI) chatbots such as Chat GPT in November 2022 has opened new pathways in learning. In 2023 and 2024, there has been a surge in their use by educators and students alike. Educators have applied GAI in areas such as assessment, feedback, content creation, and personalized learning. Concurrently, students have used these tools to assist with written assignments, generate explanations, comprehend complex topics, and review and research subjects.

This study examines the perceived effectiveness of a GPT-4-based personal study assistant, a customized version of Chat GPT, designed to supplement students' learning of specific course content. The GPT was configured with content from a tertiary-level course on intercultural communication and leadership, delivered within a postgraduate engineering degree at an Austrian university. Using Chat GPT-4’s online interface, course content, relevant literature, and information on the covered topics were uploaded. Additionally, an informal course script, available to enrolled students, and administrative information such as examinable components, assignment submission dates, and exam guidelines were provided to the GPT. The GPT was also instructed to refuse to respond to questions outside this specific scope.

During the course, students were invited to test the GPT system. A key question of this study was whether students would embrace the GPT and view it as a viable and effective tool to enhance their learning. A mixed-methods approach was employed, involving both quantitative and qualitative data collection from students (N = 34) who reported their interactions with the GPT. The findings indicate a generally positive reception of the personalized learning assistant, with most students agreeing that the assistant was helpful in clarifying course material, answering queries, and providing relevant examples. Results also suggest that students believed the GPT would be a useful revision tool for the final exam and that they would be comfortable asking questions through the GPT that they might not ask during a lecture. Key areas for improvement include better handling of off-topic questions and more accurate content alignment with course objectives. Furthermore, there was a reasonably high willingness among students to use similar GAI tools in other subjects, indicating potential for broader application.

Keywords: Artificial intelligence in education, Chat GPT, new media.