Computational and algebraic thinking are essential for problem-solving, reasoning, and generalisation in science, technology, engineering, and mathematics (STEM). Given their significance in 21st-century education, reliable assessment instruments are needed to evaluate and support students' development in these areas. To address this need, we developed COMATH, an online assessment instrument designed to measure computational and algebraic thinking competencies in students aged 9–14. The development process was grounded in established theoretical frameworks, including systematic literature reviews and cross-national curriculum analysis. It was further informed by experts from diverse cultural backgrounds, including researchers and teachers, as well as feedback from students as target users. COMATH is structured into three levels based on students' cognitive development: COMATH1 (ages 9–10), COMATH2 (ages 11–12), and COMATH3 (ages 13–14). The instrument comprises 29 test items assessing computational thinking, with 14 focusing on algorithmic thinking and 15 integrating algorithmic thinking with other computational thinking skills. Additionally, it includes 78 items evaluating six sub-skills of algebraic thinking: generalised arithmetic (25 items); equivalence, equations, and inequalities (21 items); functional thinking (8 items); representation (7 items); transformation (10 items); and transversal skills (7 items). A pilot study was conducted between autumn 2023 and spring 2024 to evaluate the effectiveness of COMATH across different age groups and educational contexts. The study involved over 3,000 students and their teachers from six countries: Finland, Hungary, Lithuania, Spain, Sweden, and Türkiye. The aim was to collect quantitative and qualitative data to refine and improve the instrument. Following the pilot, students completed an online survey, rating six statements on a five-point Likert scale regarding their experience with the online format, test difficulty, usability, and overall impressions. Similarly, teachers completed a survey assessing eight aspects of the test using a five-point Likert scale, including perceived validity, alignment with national curricula, difficulty level, ease of administration, usefulness for assessment, need for revisions, experience administering the test, and willingness to use similar assessments in the future. Teachers also provided qualitative justifications for their ratings. The insights from students and teachers will contribute to refining COMATH and informing the development of future assessment instruments for computational and algebraic thinking competencies.

Keywords: Computational thinking, algebraic thinking, online assessment, assessment instrument development, assessment instrument evaluation, cross-cultural research, teacher feedback, student feedback.