THE DEVIL IS IN THE DETAILS: DIGITAL ASSESSMENT OPPORTUNITIES AND CHALLENGES IN HIGHER EDUCATION - A CASE STUDY FROM SWEDISH ENGINEERING EDUCATION
T. Karunaratne, P. Axelsson, E. Lindblad, F. Enoksson
Higher education expects a rapid transformation as envisioned by the Digital Education Action Plan. However, contemporary research reveals a significant gap in the adoption of digital assessment methods compared to digital learning methods. While existing literature identifies various factors influencing assessment transformation, the disparity between digital adoption in learning and assessment continues to grow. We propose that the complexity of the problem is in the details. Concrete methodological evidence on how these gaps are systematically translated into actionable implementation strategies for digital assessments in highly regulated HE environments is lacking in the literature. Moreover, the adoption of digital assessment remains low, even in institutions with robust digital infrastructure, urging the need to investigate the underlying barriers. This research thus empirically explores teachers' perceived challenges and opportunities in implementing digital assessments. The methodological choice is a case study from the Swedish Engineering HE Institution. A research design of mixed methods is employed, starting with a quantitative situational analysis of the current examination landscape, which assesses the balance between traditional and digital assessments. Semi-structured interviews with bachelor’s and master’s level teachers explored the barriers and opportunities of integrating digital examinations into their courses and programs.
The situational analysis was based on scraped data about all the examinations held in the university in the previous academic year. The results showed an alarmingly low proportion of digital to traditional examinations. More than 90% of examinations were held at the end of their courses, and in examination halls under supervision, irrespective of the digital infrastructure of the university that provides many opportunities for the design and implementation of digital assessments. The 6-step thematic analysis of the interviews revealed several practical drawbacks and potential opportunities, which are summarized into a SWOT (Strengths, Weaknesses, Opportunities, Threats). The uncertainty of digital tools used for assessment appears to be a key challenge for teachers. Despite being digitally literate, teachers were concerned about the accuracy of tool customisation, the setup for specific exams, potential technology malfunctions, and the need for contingency plans in case of technical issues. This outcome highlights the importance of robust and reliable technology and the need for universities to adopt adaptable and responsive solutions. The opportunities for digital formative assessment were also highlighted. Teachers predominantly prefer using technology to continuously assess and track students' progress toward learning goals throughout the course. However, they still value a final examination at the end of the course to ensure performance quality and outcomes. While teachers appreciate human-in-the-loop in assessment and are hesitant about fully automated grading for summative assessments, they see the potential for technology to support assessments up to a passing grade.
Interestingly, they emphasize the need for sustainable rather than cutting-edge technology to ensure reliable and trustworthy digital assessments. The next phase of this ongoing study focuses on translating these findings into actionable implementation strategies and evaluating their impact.
Keywords: Digital Assessment, Assessment methods, Higher Education, Technology adoption, Digital Transformation.