The use of Artificial Intelligence (AI) Chatbots, especially that of ChatGPT, is becoming increasingly greater both in an individual level and in the context of the educational process in general. The purpose of this study is to explore the effects that the use of an artificial intelligence Chatbot has on the improvement of English Language Learners’ grammar skills in Secondary Education. The study follows a mixed methods approach. Firstly, quantitative research is conducted through teaching interventions in two separate language teaching classes of a private foreign language school, with a total of 11 B1 level students based on the CEFR (Common European Framework of Reference for Languages), with the aim of investigating and determining the possible effects of using ChatGPT in improving EFL (English as a Foreign Language) students' grammar skills. The first group of students (experimental group) engaged in a communication with ChatGPT during the lesson, while in the second group (control group) a traditional method of teaching was used. To compare the students’ improvement of grammar skills, pre and post tests were contacted and analyzed through the use of descriptive statistics analysis in EXCEL. The second part of the research, namely the qualitative research of the study, was conducted through a conversation between four English language teachers within the context of a focus group. The purpose of this discussion was to record teachers’ opinions on the use of AI Chatbots in the educational process and more specifically on how they improve grammar skills. The results showed that there is a significant improvement in the grammar skills of students who used ChatGPT compared to the other group, while the majority of teachers considered the use of Chatbots beneficial and useful in the educational process, while some maintained their doubts. Hence, the research’s findings suggest that Chatbots, specifically ChatGPT, can be used effectively for the improvement of secondary EFL students’ grammatical skills, as a feedback and information providing tool. Simultaneously, their use is being well-received not only by the students but also by the teachers. In conclusion, the analysis of the quantitative data show that ChatGPT-3.5 can be used in order to personalize and adapt the educational material with the purpose of facilitating the students and improving their grammatical skills. At the same time, the analysis of the qualitative data show that the educators support the use of Chatbots within the educational process, even though some of them are cautious as it is still considered a new tool. Ultimately, this study confirms the positive effects a chatbot can have in an educational content while suggestions for further research are being made.

Keywords: AI, ChatGPT, Grammar Skills, English Language.