Many students at Austrian universities of applied sciences seem to use AI chatbots (AIC) for many tasks they are given when learning for courses. There are several scientists from the domains of pedagogy, psychology, and STEM predicting that using AIC can lead to cognitive offloading, cognitive atrophy, and digital dementia, especially when the students don’t already have a fundamental understanding of the domain they are about to learn. Therefore, they see the risk of reduced learning capacity and diminished ability of critical thinking. In our opinion, the discussion must be intensified as soon as possible to weigh the advantages and disadvantages associated with AIC in learning.

Many professors see a widespread trend for extremely elaborate texts of students that seem to be verbalized much better than the students should be capable of. But they don’t have any possibility to prove it, nor to decide which part is written by an AIC and to which amount the expressed thoughts are genuine human. Random tests with AI detectors can never provide 100% accuracy; therefore, the professors are not able to react accordingly.

The above-described phenomena show that there is an urgent demand for discussion of assessments, grading, and definitions of quality and learning. To provide a foundation for these fundamental decisions, we plan to invoke several scientific programs that will deal with the status quo and the eligibility of available tools.
As a first step, we try to find out if the findings that statistical changes in the usage of words over time can imply widespread usage of AIC can be applied to the context of bachelor theses and master theses at Austrian universities of applied sciences.

Our test series compares more than 700 texts written by students from different study programs in the last 5 years. We searched for word clustering, changes in the usage of words over time, and usage of special phrases. Especially important was the comparison with texts from a time when no AIC was available. We randomly tested parts of eye-catching results with different AI detectors to consolidate the results and try to provoke false positives.

Even when we did not find a way to prove single texts to be AI-written, we were able to show obvious changes in style and word choice that allowed us to estimate the amount of AI usage in different study programs and cohorts.

Keywords: Technology, education, AI.