This research analyzes changes in the results of an online test conducted using the Merlin educational platform in the Colors in Multimedia Systems course at the Multimedia graduate program at the University North, Croatia. The Merlin test consists of 24 questions of various types that test not only knowledge, but also the ability to analyze and apply information. The analysis covers the last three academic years, with a special emphasis on the impact of the increasing application of artificial intelligence (AI) in the learning process. Since the test and teaching principles remained unchanged throughout the three observed academic years, the identified trend of a decrease in the number of excellent results and an increase in students in the middle scoring range (15-20 points) indicates possible changes in students' learning habits. The paper additionally explores the differences in students' online test results compared to two AI tools that solved a completely identical test. The aim was to evaluate the accuracy of answers according to the type of question, the time to solve it, and the types of errors. The results show that AI tools are very accurate for factual questions and tasks that require short answers, while students often show variations in accuracy due to subjective understanding and different learning approaches. For more complex problem-solving tasks, AI tools show limitations in logical reasoning and understanding context. The time required to solve tests is significantly shorter for AI tools, which indicates a high speed of information processing, but raises questions about the depth of understanding. Error analysis reveals that AI tools sometimes generate convincing but incorrect answers, while student errors are more often the result of a lack of knowledge or misinterpretation of questions. Although AI can facilitate the learning process, its excessive use can lead to superficial acquisition of knowledge and a decrease in analytical skills. These results indicate the need to adapt educational methods – tests should encourage the application of knowledge rather than the mere recognition of information, and AI should be used as a support, not a substitute for critical thinking.

Keywords: Education, Merlin platform, online test, Artificial intelligence, result comparison.