During the last 5 years, the usage of Artificial Intelligence (AI) in higher education has suffered a rapid increase. Incorporating AI in higher education is not only an option but also a required need to prepare new generations for the employment market in continuous evolution. In this work we wanted to analyze the current use of AI by university students and evaluated its effectiveness in a controlled learning environment. The main objective was to conduct a study on the use of generative AI in the classroom to see if it can transform the teaching of programming. We wanted to characterize the current use of AI in higher education and measure its effectiveness in university learning in an experiment aimed at students of computer engineering degrees.

On one hand, we developed an anonymous online form, with several general questions about their use they make of IA. On the other hand, we suggested that Computer Science students answer similar evaluation questions by using a variety of widespread generative AIs like ChatGPT, Gemini (Google) or Copilot (Microsoft), through an anonymous online form. These questions were characterized by short answers, for example, defining computer programming theoretical concepts, describing source code, or writing code to resolve determined problems. By examining the answers, we could verify that the students currently have not made intense use of AI in their studies. The students did not have faith in AI-generated responses, but, at the same time, they were not very critical in evaluating the answers provided.

Although the generative AI does properly assist in reinforcing the theoretical programming concepts or code explanation, it does fail in resolving complex code assignments since its solutions are usually incomplete, incorrect, or low quality. Consequently, integrating AI in lecture rooms will require a more structured and ordered way to make it more efficient and improve the learning process. This situation demands supervising the student in the AI uses in the course, including the AI in the educational material. Students must learn how to correctly evaluate the answers offered by AI to be a truly effective tool. Thus, it is crucial to assess the efficacy of integrating AI in the learning process, considering the adjustments needed to its evolution to optimize its profits.  

Further work will be done to incorporate generative AI in the classroom when teaching programming but always considering the importance of develop source code also without help.

Keywords: Generative Artificial Intelligence, Higher Education, Teaching of programming.