Critical thinking skills are essential in the age of Artificial Intelligence (AI) to ensure that students can effectively evaluate AI-generated content, avoid bias, and make informed decisions that meet ethical standards in higher education. Based on the foundational literature on critical thinking [1], the study proposes a framework for analysing and assessing critical thinking skills with a focus on the appropriate use of generative AI. The framework consists of four dimensions: 1) analytical skills: the ability to interpret and understand the output of ChatGPT, recognising biases and underlying assumptions, 2) evaluation skills: evaluating AI-generated text, data, images, 3) ethical reasoning skills: reasoning and decision making, and 4) metacognition: reflecting on personal biases, reflecting on ChatGPT use. On this basis, we developed a task-set compromising of 11 critical thinking tasks and conducted an online experiment with 39 students. The experimental group (EG, 20 students) used ChatGPT for assistance, while the control group (CG, 19 students) relied on conventional web search in Google. Subsequently, students were asked to evaluate the tasks in terms of 1) ease of understanding, 2) ease of solving, and 3) satisfaction with their own performance. The results demonstrate that the task scores of the EG are, on average, slightly higher than those of the CG. However, the results show shortcomings of the EG in the field of analytical skills, deductive reasoning and the reflection on the appropriate use of ChatGPT. The study is limited by the small sample size and further research is needed.

Keywords: Artificial Intelligence, Generative AI, critical thinking skills.