Collecting student responses in classrooms is increasingly supported by digital technologies, for example through different type of audio response systems, so called clickable mentometers. However, learning does not take place only in the classroom, and excursions to for example the theatre, the library or the museum could be considered a chance to not only learn new things but also to increase student learning by testing what they know. A need for a method to collect student responses “in the wild" is hence needed. This study investigated the possibility of using mentometers in a more informal learning setting (a museum). In more detail, we looked at students’ attitude towards using the mentometers and to what extent they actually thought through their answers before responding.

On six different occasions, a total of 131 Swedish grade 6 students participated in guided museum tours. Each student received a handheld mentometer device, as well as a short introduction on how to use it. During the tour, they were asked to respond to seven questions related to the content of the guided tour. Following the visit, a questionnaire was sent out to the students to collect data on their experiences of and attitudes towards using the mentometer. 113 students responded to the questionnaire. The teachers also provided some brief feedback on the perceived concentration level of their students.

The students’ attitudes were generally positive: 45 students responded that using the mentometers to respond to questions was fun, 42 responded that it was exciting, and 28 students found it to be something new. On the more negative side, 16 students found it annoying, 10 found it complicated, and 7 found it difficult. About half of the students (57 students) responded that they carefully reflected on a question before answering. This group of students had a mean total score of 4.6 points out of a total of 7 points (SD=1.5). The 32 students responded that they only reflected a little had a mean score of 4.0 points (SD=2.0), while 27 disclosed that they did not think about the question before anwering, or mostly guessed (M=3.6 points, SD=1,6).

To validate the students’ meta-cognitive reflection, a generalized linear mixed-effect model was used to model the probability of responding correctly as predicted by the amount of reflection students indicated that they had done prior to answering (three levels). The model predicted students that declared that they carefully reflected before responding had a 82.4% probability of answering correctly, for students that stated that they only briefly thought about the questions, the same number was 72.0%. Lastly, students that stated that they did not think about the question or mostly guessed had a probability 58.6% to answer correctly. The fixed effect of level of reflection was significant (<0.005).

For two classes, the teacher responded that using the mentometers seemed to affect the concentration of the students’ slightly, while for four classes, the teachers stated that they did not seem to affect the concentration levels at all.

The pilot study indicates that handheld devices, such as mentometers, can successfully be used for collecting responses in the wild, such as in a museum, which may be increasingly important as learning is moving beyond the classroom. In addition, the results also suggest that mentometers may be a promising method to study transfer effects outside of the classroom.

Keywords: Learning assessment, digital tools, response systems.