The process of teaching and learning is rapidly changing following the development of Information Technologies in the direction of online classrooms. One of the pillars of the learning process is verification of learning outcomes fulfillment; both students and educators rely on it, first for self-evaluation, and the later for evaluation. There exist platforms for automated questions’ generation, but none of them in Bosnian language; furthermore, as the grammar is specific, automated translation can result in meaningless or statements with modified meaning, which would be useless in the sensitive process of evaluation. For these reasons there arises the need to develop a system that will enable personalized automated generation of questions from electronic teaching materials for Bosnian language speakers.

The system envisioned at the Faculty of Information Technologies (FIT), University “Dzemal Bijedic” in Mostar, Bosnia and Herzegovina, is foreseen to be usable at all education levels and is due to follow pedagogical standards. To enable different levels of difficulty for the automatically generated questions, apart from the topic or content, the authors also propose to consider the distance among the listed answers. In this paper, the developed methodology is tested on a set of questions related to the historical events, as a part of the project with the Centre for Peace and Multiethnic Reconciliation Mostar (CZM). The aim of the project is to model the platform for the automated and personalized verification of learning outcomes fulfillment in Bosnian language.

The model testing procedure is the following: 50 IT students fill in the questionnaire on two groups of questions, manually developed and automatically generated. Each student estimates her/his knowledge of the matter (general history knowledge) before and after taking the test. In addition to this, students answer the questions and evaluate them regarding the following: clarity or formulation, difficulty, and informativity. The survey data were analyzed in MS Excel with the hypothesized statistically significant difference between the two groups. Data analysis shows no statistically significant difference, supporting the fact that automatically generated questions in the Bosnian language are clearly defined, as well as properly matched with difficulty level to serve as an accurate tool for learning outcomes’ evaluation and verification.

Keywords: Learning outcomes verification, automation, standardization.