This paper offers a comparative analysis of ChatGPT, a sophisticated conversational artificial intelligence model; in automated assessment of programming processes, and its alignment with human evaluation. This research was prompted due to an increasing interest for AI tools to find their way into educational assessments; particularly in large-scale programming courses where manual grading can be time-consuming and inconsistent. For this study, we will focus on two programming languages which are part of many institutions' curricula, Java and C. Participants were sent a range of online forms, including multiple-choice, code writing, and conceptual questions.

The ChatGPT responses were processed using a number of defined prompts requesting it evaluate, justify, flag, and transform errors including recommendations, that were then utilized to produce human assessments, which were labelled by the instructors using the standard prescribed education rubric, whereas the study aims to confirm the level of agreement between both the AI and human scores, as well as similarities and differences, and properties and potential use of the critiques suggested by ChatGpt.

The outcome of this study will show how AI grading systems could have the potential to be used to support or complement human scoring efforts when the need for a fast, scalable and consistent assessment is warranted. The results will help contribute to the larger conversation and provide insights regarding the fitness for purpose, reliability, fairness, and learning value associated with employing AI in educational contexts.

Keywords: ChatGPT, automated grading, programming education, artificial intelligence in education, human vs AI evaluation, Java, Python, C, educational assessment, feedback systems, scalable evaluation.