Artificial intelligence (AI) is reshaping the landscape of higher education, particularly in programming courses. The emergence of large language models (LLMs) like ChatGPT and Copilot has significantly altered how students engage with learning materials, extending the diversity of sources of assistance to include AI-driven tools alongside official documentation and community forums. While these technologies provide immediate and interactive support, they also introduce new challenges regarding academic integrity, problem-solving autonomy, and the critical assessment of AI-generated content. As educators, adapting teaching methodologies to incorporate AI effectively without undermining fundamental learning objectives is crucial.

This pilot study investigates the impact of generative AI on programming education by comparing AI-assisted learning with traditional methods. Conducted in an undergraduate programming course, the project aims to identify possible differences in students' comprehension and efficiency. The intervention consisted of a structured classroom experiment where students were randomly divided into two groups: one using conventional resources such as course materials, online forums, bibliography and documentation, and another relying exclusively on AI assistance for problem-solving. After providing a brief introduction on how to efficiently use each kind of resource, a set of programming exercises were completed under controlled conditions, with pre- and post-intervention assessments used to measure learning gains and efficiency in task completion. Complementary questionnaires were also used to quantify self-perceived competence in using AI tools and student preferences towards each methodology.

The results indicate that students using AI assistance achieved a significantly higher learning gain compared to those using traditional resources. Additionally, the AI-assisted group demonstrated improved efficiency, completing the programming exercises in 16% less time than the control group, on average. These findings suggest that generative AI can facilitate more rapid task completion while enhancing conceptual understanding.

Beyond performance metrics, student perceptions of AI integration in learning were largely positive. Participants in the AI-assisted group reported higher satisfaction and a stronger sense of engagement compared to their peers in the traditional group. In addition, most students expressed a preference for AI-based assistance over bibliography and other online resources, only surpassed by the use of course materials. On the other hand, qualitative feedback also highlighted concerns about over-reliance on AI-generated solutions.

Despite the limitations of the study, the results underscore the need for a balanced integration of AI into programming education. While these technologies can serve as powerful learning aids, their use should be carefully designed to reinforce problem-solving skills rather than replace them.

This work contributes to the ongoing discussion on AI in education, offering insights into how generative AI can be harnessed effectively to improve learning outcomes while maintaining the integrity of foundational programming instruction.

Keywords: Artificial Intelligence, ChatGPT, AI-Assisted Learning, Computer Programming.