Stay Vigilant: The Threat of a Replication Crisis in VR Locomotion Research
The ability to reproduce previously published research findings is an important cornerstone of the scientific knowledge acquisition process. However, the exact details required to reproduce empirical experiments vary depending on the discipline. In this paper, we summarize key replication challenges as well as their specific consequences for VR locomotion research. We then present the results of a literature review on artificial locomotion techniques, in which we analyzed 61 papers published in the last five years with respect to their report of essential details required for reproduction. Our results indicate several issues in terms of the description of the experimental setup, the scientific rigor of the research process, and the generalizability of results, which altogether points towards a potential replication crisis in VR locomotion research. As a countermeasure, we provide guidelines to assist researchers with reporting future artificial locomotion experiments in a reproducible form.
Best Paper Award!
@inproceedings{10.1145/3611659.3615697,
author = {Zielasko, Daniel and Weissker, Tim},
title = {Stay Vigilant: The Threat of a Replication Crisis in VR Locomotion Research},
year = {2023},
isbn = {9798400703287},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3611659.3615697},
doi = {10.1145/3611659.3615697},
abstract = {The ability to reproduce previously published research findings is an important cornerstone of the scientific knowledge acquisition process. However, the exact details required to reproduce empirical experiments vary depending on the discipline. In this paper, we summarize key replication challenges as well as their specific consequences for VR locomotion research. We then present the results of a literature review on artificial locomotion techniques, in which we analyzed 61 papers published in the last five years with respect to their report of essential details required for reproduction. Our results indicate several issues in terms of the description of the experimental setup, the scientific rigor of the research process, and the generalizability of results, which altogether points towards a potential replication crisis in VR locomotion research. As a countermeasure, we provide guidelines to assist researchers with reporting future artificial locomotion experiments in a reproducible form.},
booktitle = {Proceedings of the 29th ACM Symposium on Virtual Reality Software and Technology},
articleno = {39},
numpages = {10},
keywords = {Reproducibility, Virtual Reality, Replication Crisis, Teleportation, Locomotion, Steering},
location = {Christchurch, New Zealand},
series = {VRST '23}
}
