OPTIMIZING LANGUAGE ANALYSIS: A COMPARATIVE STUDY OF ADVANCED DIGITAL TOOLS IN TEACHER TRAINING CONTEXTS
K. Florou
Natural Language Processing (NLP), as defined by Lindy (2001), is a computerized approach for text analysis grounded in both theoretical frameworks and technological implementations. These techniques not only expand user access to information, but also introduce new paradigms for utilizing information-system services. The convergence of artificial intelligence and linguistic studies has further cemented the relevance of NLP, particularly in applications such as language recognition, syntactic analysis, and automated translation. Emerging Generative AI (GenAI) tools, which inherently function as NLP systems, represent the latest innovation in this domain. This study aims to investigate the potential benefits for university students and prospective language educators when comparing traditional NLP tools with contemporary GenAI tools for foreign language teaching. Specifically, this research focuses on assessing the comparative effectiveness of ChatGPT4.0 in language analysis tasks (Božić & Poola, 2023). The current study builds upon prior research that involved students conceptualizing instructional activities using advanced NLP tools, such as Voyant Tools (Sinclair & Rockwell, 2016), to analyze a selected text. Students then applied a similar analytical process using ChatGPT3.5 on the same text, enabling a comparative evaluation of their outcomes. In the context of big data, traditional NLP tools are designed to efficiently process extensive corpora, presenting a significant advantage; however, the limitations of the free version of ChatGPT3.5, which cannot process lengthy texts, restrict the scope of comparative analysis to shorter literary passages, such as poems or brief excerpts from novels. Following the release of ChatGPT4.0, the same cohort of prospective Italian language educators engaged in further comparisons between GenAI and traditional NLP tools, in particular, Voyant tools. In this study, participants used ChatGPT4.0 to analyze both long and short literary texts, exploring the conditions under which GenAI tools yield optimal results. The Corpus analyzed was in Italian and included four short stories by Andrea Camilleri, adding practical value to the findings in the context of Italian language teaching. The analysis that future teachers are required to conduct is lexical; specifically, they were tasked with verifying the accuracy of frequency lists generated by the two tools. After conducting the analyses, participants completed a questionnaire, which provided qualitative and quantitative insights into the tools' effectiveness. Findings indicate that the specialized functionalities of Voyant Tools enable comprehensive analysis, allowing researchers to identify patterns, trends, and nuanced linguistic features within large datasets. This study thus highlights the relative strengths and weaknesses of GenAI and NLP tools in language education, providing valuable insights for their application in teaching methodologies.
References:
[1] Božić V, & Poola I. (2023). Chat GPT and education. Preprint.
[2] Liddy, E.D. (2001). Natural Language Processing. In Encyclopedia of Library and Information Science, 2nd Ed. NY. Marcel Decker, Inc.
[3] Sinclair, S., & Rockwell, G. (2016). Embedding Voyant Tools.
Keywords: Italian Language Teachers' Training, Natural Language Processing, GenAI in FLT.