Artificial intelligence (AI) generative writing tools such as ChatGPT are being used by students to complete assignments, regardless of whether such tools have been approved for use in their respective courses or educational institutions. In this research, our aim was to determine the effectiveness of such AI generative tools via several metrics, for example, quality of writing, vocabulary, grammatical correctness, and accuracy. We assigned writing projects of biographies in three courses with different sets of students ― a first-year programming fundamentals course (Ada Lovelace), a second-year computer organization and architecture course (Steve Jobs), and a fourth-year course in entrepreneurship and management of IT systems (Paul Allen) of the Bachelor of Science in Computer Science program. In each course, half the students were assigned to write without using AI tools, while the others were required to use ChatGPT to complete the assignment. In addition to generating the essay, the students using generative AI writing tools were required to fact check the writing produced by ChatGPT and provide a list of errors. They were then asked to improve ChatGPT’s output. The course lecturers (authors of this paper), then evaluated the writing projects and compared the results of the two halves of each of the three classes. Although the findings varied per course, ChatGPT generated essays were generally of high quality, presented a coherent narrative of the biography and contained no vocabulary or grammatical errors, in comparison to student created essays. However, ChatGPT generated essays contained several historical inaccuracies, dramatic and speculative language, exaggeration of contributions that understate or overstate achievements, and repetition of a similar structure and ideas across essays. Student generated essays on the other hand used basic to intermediate vocabulary, have relatively poor grammar and sentence structure, and are less coherent than ChatGPT generated essays. Based on these findings, we believe that educators need to shift their focus from banning AI tools outright, to encouraging students to focus on fact-checking, critical thinking, and elimination of inaccuracies in their writing. Adapting assessment strategies to embrace AI will prepare students for a future where AI is an integral part of professional work, while maintaining academic standards and integrity.

Keywords: Artificial Intelligence tools, ChatGPT, Generative Writing Tools, Computing Curricula.