In recent years, tablet devices have come to be commonly used by students in their daily learning activities at school. In Japan, all students in elementary and secondary education have their own tablet devices as an everyday learning tool, just like a pencil or notebook. We have been conducting research focusing on supporting learning and assessment by using photos that capture students’ learning activities and records.

Meanwhile, generative AI has been attracting considerable attention for its rapid development and its ability to interpret images and generate textual descriptions.
We conceived that if a system could use generative AI to understand the content of the marker-circled areas in photos and generate reflection prompts for students to explain why they circled those parts, it could facilitate their reflection on learning. In this way, students’ reflective learning journals can be accumulated as learning evidence along with photos, enabling not only teachers but also students themselves to identify their strengths, potential, and progress in their own learning.

In this study, we developed a photo-based reflection support system using generative AI to extract information from marker-circled areas in a photo.
This system supports photo-based learning reflection by extracting information from marker-circled areas in photos and generating tailored “reflection prompts” on the basis of the extracted information and the type of photo. These prompts facilitate reflection, and the reflections are accumulated as reflective learning journals.

The developed system consists of three functions:
- Function for classifying photos: The system uses generative AI to classify whether a photo contains a red or blue marker-circled area.
- Function for extracting information from marker-circled areas in photos: The system uses generative AI to identify the marker-circled areas in a photo, describe the content within those areas, and extract information from them.
- Function for generating reflection prompts: This system uses generative AI to generate reflection prompts by revising predefined example reflection prompts in accordance with the classification of the photo and the information extracted from the marker-circled areas.

The process of photo-based learning reflection using the developed system is as follows:
Students take photos of learning activities or records. At the end of a lesson, they select a representative photo, circle the part that represents something they think they did well or something new they have acquired, and use the photo to reflect on their learning. They then upload the selected photo. The system classifies the type of photo on the basis of the color of the marker. The system then extracts information from the marker-circled areas and generates a reflection prompt tailored to both the type of photo and the extracted information. Students reflect on their learning on the basis of the reflection prompt provided by the system, and write their reflections into the system.

We verified the functionality of the function for extracting information from marker-circled areas in a photo and the function for generating reflection prompts. As a result, the system was generally able to extract information from the marker-circled areas, suggesting it has the potential to generate appropriate reflection prompts tailored to the extracted information.

In the future, we plan to evaluate the effectiveness of the developed system.

Keywords: Photo-based learning, photo-based reflection, reflection prompts, one-to-one tablet, generative AI, marker-circled area description.