RETHINKING MULTIPLE-CHOICE EXAMS: THE IMPACT OF NLP AI MODELS FOR ONLINE ASSESSMENT
W. Earle, A. Singh
This study explores the efficacy of Natural Language Processing (NLP) AI models, specifically ChatGPT and MS Copilot, in answering multiple-choice questions (MCQs) within the framework of Cisco Certified Network Associate (CCNA) exams. Our methodology involved presenting these AI systems with a series of exam questions and assessing their responses against a predetermined passing grade threshold. The results demonstrated that the AI-generated answers closely aligned with the expected correct responses, suggesting a high degree of accuracy and reliability.
We delve into the broader implications of these findings, discussing the potential applications of AI in enhancing exam preparation, automated grading, and personalized learning experiences. Furthermore, we address the limitations and ethical considerations inherent in deploying NLP AI for educational purposes, such as the potential for over-reliance on technology, issues of academic integrity, and the need for transparency in AI-generated assessments. Our study underscores the significant potential of AI technologies to transform educational assessments and support students and educators alike.
Keywords: Natural Language Processing, Multiple-Choice Questions, AI Assistance, Educational Assessment, Cisco CCNA, ChatGPT, MS Copilot, Ethical Considerations.