A crucial educational goal for effective teaching in 21st-century classrooms is to infuse diagnostic assessment tasks into students’ personalised learning for formative purposes. To achieve this goal, teachers should be competent to design assessment tasks and provide students with timely and constructive feedback. We refer to this competence as “assessment literacy” (AL).

The present project aims to develop pre-service science teachers’ AL through a systematic approach. The project consists of three stages:
(1) The project team organised and conducted a series of workshops to provide participants with sufficient knowledge of the conceptual change, metacognition and application of multiple-choice questions (MCQs) for diagnostic purposes;
(2) The participants were grouped and intensively shadowed by mentors to develop high-quality MCQs with written feedback and produce well-planned short teaching videos; and
(3) The participants were guided to reflect on assessor identity after studying the feedback from the secondary school teachers and students who used the developed materials in their learning and teaching.

Through active and collaborative engagement in the project, participants developed their AL in practice:
(1) They are able to develop valid MCQs regarding specific learning objectives;
(2) They developed written feedback for each distractor to induce the learner’s conceptual change and metacognition;
(3) They developed short teaching videos to explain the concepts involved in the selected MCQs; and
(4) The project developed an open online platform that included all the MCQs, feedback, and videos for secondary school teachers and students.

To analyse the project’s impacts on participants’ AL, we developed a 20-item survey instrument that assesses participants’ perceptions of their AL. The instrument is composed of 20 items that cover five aspects: valid design, reliable scoring, data analysis, fair assessment, and student voice and choice. Pre-test and post-test were administered, where participants self-rated their views on a 5-point Likert scale on the items.

Concrete examples will be provided during the conference and in the paper to illustrate how the project progressively develops participants’ AL. Furthermore, the survey results will be presented and discussed. Lastly, the implications for formative assessment practice will be explored.

Keywords: Assessment literacy, diagnostic assessment, chemistry education, alternative conception, metacognition.