The use of Virtual Reality (VR) in educational contexts is increasing, due to the decrease and availability of hardware costs. The technology lets researchers and educators design environments that are pedagogically sound, containing features that would not be feasible or realistic in real-life based instructional designs, such as visualization of otherwise invisible forces, reflective pausing in high-stress environments, or simulation-based learning using dangerous or expensive materials. However, evaluation of software, with regard to the learning outcomes, should be conducted before implementing it in educational contexts.

The current between-subjects pilot study compares the learning outcomes of failing and passing students in three physics classes (N=39 students) in a Swedish high school, after completing a lab session on Archimedes’ principle, either in VR or the equivalent lab in real-life. The software used in this study is designed based on instructional principles, explicitly guided by the cognitive load theory, an instructional design theory that can guide the design of learning environments based on underlying cognitive architecture of humans. Before conducting the lab session, all students listened to a lecture on the topic. Students in both conditions were matched based on previous physics grade. Performance was measured as the number of correct responses in a post-test containing 9 questions on the principle, from recall to transfer questions, that had previously been tested and evaluated.

Of students that were reported by their teacher to be failing, or in the risk of failing their physics class (N=17), the mean score on the post-test was 4.44 (SD=1.13) (out of 9) possible correct answers in the virtual condition. In the real-life condition, the mean score was 5.00 (SD=1.93). In the group of passing students (N=22), students in the virtual condition scored 4.27 (SD=2.15), while students completing the real-life lab had a mean score of 5.54 (SD=1.57). A multiple linear regression was fit to the data. Outcome measure was post-test score, and the predictors were condition (VR/R) as well as previous grade (pass/fail). The model intercept (for real-life condition with failing students) was 4.25, and the performance was predicted to increase by 0.96 points in the virtual condition (p=0.09). For passing students, the score was predicted to be 0.18 points higher than failing students (p=0.76).

Despite the statistical power of the model being very low, some trends can be seen in the data that are worth exploring further. The difference between failing and passing students was non-significant. Nevertheless, students passing their physics class seem to perform slightly better than failing students, as was expected.

Irrespective of previous grade, students in the virtual condition seem to perform better than students in the real-life equivalent. The instructional design of the VR software may partly explain these results, as the modality (VR) itself allowed for design features, such as unlimited variation of crucial variables (such as density of fluid), as well as visualization of “invisible” forces on each object.

Possible methodological limitations, and further improvements for comparative research in VR, such as using recordings of interactions with the environments, are discussed.

Keywords: Virtual Reality, educational technology, physics, STEM.