Embodied Self-Motion Illusions in VR

profile

How can we pro­vide humans with a believ­able sen­sa­tion of being in and moving through computer-generated envi­ron­ments (like VR, com­puter games, or movies) with­out the need for costly and cum­ber­some motion plat­forms or large free-space walk­ing areas? That is, how can we “cheat intel­li­gently” by pro­vid­ing a com­pelling, embod­ied self-motion illu­sion (“vec­tion”) with­out the need for full phys­i­cal motion?

In a series of exper­i­ments, we inves­ti­gated the con­tri­bu­tion and inter­ac­tion of dif­fer­ent sen­sory modal­i­ties as well as higher-level, cog­ni­tive influences.

Below is a selec­tion of such work [more will come as the web­site evolves]

Spatialized Sound Enhances Biomechanically-Induced Self-Motion Illusion (Vection)

The use of vec­tion, the illu­sion of self-movement, has recently been explored as a novel way to immerse observers in medi­ated envi­ron­ments through illu­sory yet com­pelling self-motion with­out phys­i­cally moving. This pro­vides advan­tages over exist­ing sys­tems that employ costly, cum­ber­some, and poten­tially haz­ardous motion plat­forms, which are often sur­pris­ingly inad­e­quate to pro­vide life-like motion expe­ri­ences. This study inves­ti­gates whether spa­tial­ized sound rotat­ing around the sta­tion­ary, blind­folded lis­tener can facil­i­tate bio­me­chan­i­cal vec­tion, the illu­sion of self-rotation induced by step­ping along a rotat­ing floor plate. For the first time, inte­grat­ing simple audi­tory and bio­me­chan­i­cal cues for turn­ing in place evoked con­vinc­ing cir­cu­lar vec­tion. In an audi­tory base­line con­di­tion, par­tic­i­pants expe­ri­enced only spa­tial­ized audi­tory cues. In a purely bio­me­chan­i­cal con­di­tion, seated par­tic­i­pants stepped along side­ways on a rotat­ing plate while lis­ten­ing to mono mask­ing sounds. Scores of the bi-modal con­di­tion (bin­au­ral bio­me­chan­i­cal cues) exceeded the sum of both single cue con­di­tions, which may imply super-additive or syn­er­gis­tic effects.

YouTube Preview Image

Stepping along a rotat­ing cir­cu­lar tread­mill while being blind­folded can (after a few sec­onds) create a com­pelling & embod­ied illu­sion of rotat­ing in the oppo­site direc­tion[ERROR: No URL was passed to the generic video BBCode]. Studying such “biomechnically-induced cir­cu­lar vec­tion” can help us to  better under­stand human multi-modal infor­ma­tion pro­cess­ing and even­tu­ally help us to improve motion sim­u­la­tion in Virtual Reality.

 

YouTube Preview Image

First demo of our new cir­cu­lar tread­mill in the iSpace lab. Since then the setup has had some major updates.

 

Why not build an omni-directional tread­mill so people can walk in all direc­tion in the Virtual Environment? While recent progress has been amaz­ing, such omni-directional tread­mills are still huge, loud, expen­sive, and need many safety mea­sures (see video below for an exam­ple from the Cyberwalk project). In addi­tion, it seems that walking-in-place can only reli­ably induce cir­cu­lar vec­tion, but not translational/linear vec­tion. Hence, we decided to focus on using out under­stand­ing if human multi-modal cue inte­gra­tion to “cheat intel­li­gently” and aiming for a com­pelling embod­ied illu­sion of self-motion (i.e., “per­cep­tual realism/effectiveness”) instead of aiming for biomechanical/stimulus realism.

YouTube Preview Image

Media Gallery

Related Publications

Feuereissen, D. 2008. “VR: Getting the Reality Part Straight – Does Jitter and Suspension of the Human Body Increase Auditory Circular Vection?” Bachelor’s Thesis, Department of Computer Science in Media. http://www.kyb.mpg.de/publication.html?publ=5071.
Palmisano, Stephen, Shinji Nakamura, Robert S. Allison, and Bernhard E. Riecke. 2020. “The Stereoscopic Advantage for Vection Persists Despite Reversed Disparity.” Attention, Perception, & Psychophysics 82 (4): 2098–2118. https://doi.org/10.3758/s13414-019–01886-2).
Keshavarz, Behrang, Aaron E. Phillip-Muller, Wanja Hemmerich, Bernhard E. Riecke, and Jennifer J. Campos. 2018. “The Effect of Visual Motion Stimulus Characteristics on Vection and Visually Induced Motion Sickness.” Displays 58 (July):71–81. https://doi.org/10.1016/j.displa.2018.07.005.
Palmisano, Stephen, and Bernhard E. Riecke. 2018. “The Search for Instantaneous Vection: An Oscillating Visual Prime Reduces Vection Onset Latency.” PLOS ONE 13 (5): 1–26. https://doi.org/10.1371/journal.pone.0195886.
Hashemian, A. M., Kitson, A., Nquyen-Vo, T., Benko, H., Stuerzlinger, W., & Riecke, B. E. (2018). Investigating a Sparse Peripheral Display in a Head-Mounted Display for VR Locomotion. 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 571–572. https://doi.org/10.1109/VR.2018.8446345
Mursic, Rebecca A., Bernhard E. Riecke, Deborah Apthorp, and Stephen Palmisano. 2017. “The Shepard-Risset Glissando: Music That Moves You.” Experimental Brain Research 235 (10): 3111–27. https://doi.org/10.1007/s00221-017‑5033-1.
Kruijff, Ernst, Alexander Marquardt, Christina Trepkowski, Robert Lindemann, Andre Hinkenjann, Jens Maiero, and Bernhard E. Riecke. 2016. “On Your Feet! Enhancing Self-Motion Perception in Leaning-Based Interfaces through Multisensory Stimuli.” In Proceedings of ACM Symposium on Spatial User Interaction (SUI ’16), 149–58. Tokyo, Japan: ACM. https://doi.org/10.1145/2983310.2985759.
Riecke, Bernhard E. 2016. “Using Spatialized Sound to Enhance Self-Motion Perception in Virtual Environments and Beyond: Auditory and Multi-Modal Contributions.” Canadian Acoustics 33 (3): 148–49.
Keshavarz, Behrang, Bernhard E. Riecke, Lawrence J. Hettinger, and Jennifer L. Campos. 2015. “Vection and Visually Induced Motion Sickness: How Are They Related?” Frontiers in Psychology 6 (413): 1–11. https://doi.org/10.3389/fpsyg.2015.00413.
Riecke, Bernhard E., Daniel Feuereissen, John J. Rieser, and Timothy P. McNamara. 2015. “More than a Cool Illusion? Functional Significance of Self-Motion Illusion (Circular Vection) for Perspective Switches.” Frontiers in Psychology 6 (1174): 1–13. https://doi.org/10.3389/fpsyg.2015.01174.
Riecke, Bernhard E., Jacob Freiberg, and Timofey Y. Grechkin. 2015. “Can Walking Motions Improve Visually Induced Rotational Self-Motion Illusions in Virtual Reality?” Journal of Vision 15 (2): 1–15. https://doi.org/10.1167/15.2.3.
Riecke, Bernhard E., and Jacqueline D. Jordan. 2015. “Comparing the Effectiveness of Different Displays in Enhancing Illusions of Self-Movement (Vection).” Frontiers in Psychology 6 (713). https://doi.org/10.3389/fpsyg.2015.00713.
Riecke, B. E., & Schulte-Pelkum, J. (2015). An Integrative Approach to Presence and Self-Motion Perception Research. In F. Biocca, J. Freeman, W. IJsselsteijn, M. Lombard, & R. J. Schaevitz (Eds.), Immersed in Media: Telepresence Theory, Measurement and Technology (pp. 187–235). Springer. doi: 10.1007/978–3-319–10190-3_9.
Lawson, B. D., & Riecke, B. E. (2014). The Perception of Body Motion. In K. S. Hale & K. M. Stanney (Eds.), Handbook of Virtual Environments: Design, Implementation, and Applications (2nd ed., pp. 163–195). CRC Press.
Riecke, Bernhard E., Daniel Feuereissen, John J. Rieser, and Timothy P. McNamara. 2014. “Can Self-Motion Illusions (Circular Vection) Facilitate Spatial Updating?” Poster pre­sented at the Spatial Cognition 2014 Conference, Bremen, Germany. http://conference.spatial-cognition.de/SC2014/.
Riecke, B. E, Jacqueline D. Jordan, Mirjana Prpa, and Daniel Feuereissen. 2014. “Underlying Perceptual Issues in Virtual Reality Systems: Does Display Type Affect Self-Motion Perception?” Talk pre­sented at the 55th Annual Meeting of the Psychonomic Society (Psychonomics), Los Angeles, USA.
Seno, Takeharu, Stephen Palmisano, Bernhard E. Riecke, and Shinji Nakamura. 2014. “Walking with­out Optic Flow Reduces Subsequent Vection.” Experimental Brain Research 233 (1): 275–81. https://doi.org/doi:10.1007/s00221-014‑4109-4.
Riecke, Bernhard E. 2013. “Creating a Moving Experience  with­out Moving the Observer: Perceptual Aspects & Display Factors in Immersive Virtual Reality (and Elsewhere).” Invited Talk pre­sented at the International Toronto Stereoscopic Film Conference, Toronto, Canada. https://youtu.be/98hq_a_nvso.
Riecke, B. E., & Schulte-Pelkum, J. (2013). Perceptual and Cognitive Factors for Self-Motion Simulation in Virtual Environments: How Can Self-Motion Illusions (“Vection”) Be Utilized? In F. Steinicke, Y. Visell, J. Campos, & A. Lécuyer (Eds.), Human Walking in Virtual Environments (pp. 27–54). Springer. doi: 10.1007/978–1-4419–8432-6_2.
Freiberg, Jacob, and Bernhard E. Riecke. 2013. “Its Your Turn: Enhancing Visually Induced Self Motion Illusions (‘Vection’) with Walking Motions in Virtual Reality.” Poster pre­sented at the 54th Annual Meeting of the Psychonomic Society (Psychonomics), Toronto, Canada.
Riecke, Bernhard E., and Daniel Feuereissen. 2012. “To Move or Not to Move: Can Active Control and User-Driven Motion Cueing Enhance Self-Motion Perception (‘Vection’) in Virtual Reality?” In ACM Symposium on Applied Perception SAP, 17–24. Los Angeles, USA: ACM. https://doi.org/10.1145/2338676.2338680.
Riecke, Bernhard E., Salvar Sigurdarson, and Andrew P. Milne. 2012. “Moving Through Virtual Reality Without Moving?” Cognitive Processing 13 (1): 293–97. https://doi.org/10.1007/s10339-012‑0491-7.
Riecke, B. E. (2011). Compelling Self-Motion Through Virtual Environments Without Actual Self-Motion – Using Self-Motion Illusions (“Vection”) to Improve User Experience in VR. In J. Kim (Ed.). In J.-J. Kim (Ed.), Virtual Reality (pp. 149–176). InTechOpen. https://www.intechopen.com/books/virtual-reality/compelling-self-motion-through-virtual-environments-without-actual-self-motion-using-self-motion-ill
Riecke, Bernhard E, Daniel Feuereissen, John J Rieser, and Timothy P McNamara. 2011. “Spatialized Sound Enhances Biomechanically-Induced Self-Motion Illusion (Vection).” In Proceedings of the 2011 Annual Conference on Human Factors in Computing Systems, 2799–2802. CHI ’11. Vancouver, Canada. https://doi.org/10.1145/1978942.1979356.
Riecke, Bernhard E, Daniel Feuereissen, and John J Rieser. 2010. “Spatialized Sound Influences Biomechanical Self-Motion Illusion (‘vec­tion’).” In Proceedings of the 7th Symposium on Applied Perception in Graphics and Visualization, 158–158. APGV ’10. https://doi.org/10.1145/1836248.1836280.
Riecke, Bernhard E., Aleksander Väljamäe, and Jörg Schulte-Pelkum. 2009. “Moving Sounds Enhance the Visually-Induced Self-Motion Illusion (Circular Vection) in Virtual Reality.” ACM Transactions on Applied Perception (TAP) 6 (March):7:1–7:27. https://doi.org/10.1145/1498700.1498701.
Riecke, Bernhard E. 2009. “Cognitive and Higher-Level Contributions to Illusory Self-Motion Perception (‘Vection’): Does the Possibility of Actual Motion Affect Vection?” Japanese Journal of Psychonomic Science 28 (1): 135–39. http://ci.nii.ac.jp/naid/110007482465.
Riecke, B. E., D. Feuereissen, and J. J. Rieser. 2009. “Rotating Sound Fields Can Facilitate Biomechanical Self-Motion Illusion (‘cir­cu­lar Vection’).” Journal of Vision 9 (8): 714–714. https://doi.org/10.1167/9.8.714.
Riecke, Bernhard E., Daniel Feuereissen, and John J Rieser. 2009. “Auditory Self-Motion Simulation Is Facilitated by Haptic and Vibrational Cues Suggesting the Possibility of Actual Motion.” ACM Transactions on Applied Perception (TAP) 6:20:1–20:22. https://doi.org/10.1145/1577755.1577763.
“Contribution and Interaction of Auditory and Biomechanical Cues for Self-Motion Illusions (‘cir­cu­lar Vection’).” 2008. Poster pre­sented at the CyberWalk work­shop, Tübingen, Germany.
Riecke, Bernhard E, Daniel Feuereissen, and John J Rieser. 2008. “Auditory Self-Motion Illusions (‘cir­cu­lar Vection’) Can Be Facilitated by Vibrations and the Potential for Actual Motion.” In Proceedings of the 5th Symposium on Applied Perception in Graphics and Visualization, 147–54. APGV ’08. https://doi.org/10.1145/1394281.1394309.
Riecke, Bernhard E., D. W. Cunningham, and H. H. Bülthoff. 2007. “Spatial Updating in Virtual Reality: The Sufficiency of Visual Information.” Psychological Research 71 (3): 298–313. https://doi.org/10.1007/s00426-006‑0085-z.
Schulte-Pelkum, J. 2007. “Perception of Self-Motion: Vection Experiments in Multi-Sensory Virtual Environments.” PhD thesis, Ruhr-Universität Bochum. https://hss-opus.ub.ruhr-uni-bochum.de/opus4/frontdoor/deliver/index/docId/2735/file/diss.pdf.
Riecke, Bernhard E., Jörg Schulte-Pelkum, Marios N Avraamides, Markus Von Der Heyde, and Heinrich H Bülthoff. 2006. “Cognitive Factors Can Influence Self-Motion Perception (Vection) in Virtual Reality.” ACM Transactions on Applied Perception (TAP) 3 (July):194–216. https://doi.org/10.1145/1166087.1166091.
Riecke, B. E. 2006. “Simple User-Generated Motion Cueing Can Enhance Self-Motion Perception (Vection) in Virtual Reality.” In Proceedings of the ACM Symposium on Virtual Reality Software and Technology, 104–7. Limassol, Cyprus: ACM. https://doi.org/10.1145/1180495.1180517.
Riecke, B. E., & Schulte-Pelkum, J. (2006). Using the per­cep­tu­ally ori­ented approach to opti­mize spa­tial pres­ence & ego-motion sim­u­la­tion (No. 153). MPI for Biological Cybernetics. http://www.kyb.mpg.de/publication.html?publ=4186
Riecke, B. E., J. Schulte-Pelkum, and F. Caniard. 2006. “Visually Induced Linear Vection Is Enhanced by Small Physical Accelerations.” Poster pre­sented at the 7th International Multisensory Research Forum (IMRF), Dublin, Ireland.
Mohler, Betty J, William B Thompson, Bernhard Riecke, and Heinrich H Bülthoff. 2005. “Measuring Vection in a Large Screen Virtual Environment.” In Proceedings of the 2nd Symposium on Applied Perception in Graphics and Visualization, 103–9. APGV ’05. https://doi.org/http://doi.acm.org/10.1145/1080402.1080421.
Riecke, Bernhard E, Jörg Schulte-Pelkum, Marios N Avraamides, Markus von der Heyde, and Heinrich H Bülthoff. 2005. “Scene Consistency and Spatial Presence Increase the Sensation of Self-Motion in Virtual Reality.” In Proceedings of the 2nd Symposium on Applied Perception in Graphics and Visualization, 111–18. APGV ’05. https://doi.org/10.1145/1080402.1080422.
Riecke, B. E., J. Schulte-Pelkum, F. Caniard, and Heinrich H. Bülthoff. 2005. “Auditory Cues Can Facilitate the Visually-Induced Self-Motion Illusion (Circular Vection) in Virtual Reality.” Poster pre­sented at the 8. Tübingen Perception Conference (TWK), Max Planck Institute for Biological Cybernetics, Germany.
Riecke, B. E., Schulte-Pelkum, J., Caniard, F., & Bülthoff, H. H. (2005). Spatialized audi­tory cues enhance the visually-induced self-motion illu­sion (cir­cu­lar vec­tion) in Virtual Reality. (No. 138). MPI for Biological Cybernetics. http://www.kyb.mpg.de/publication.html?publ=4187
Riecke, B. E., J. Schulte-Pelkum, F. Caniard, and H. H Bülthoff. 2005. “Influence of Auditory Cues on the Visually-Induced Self-Motion Illusion (Circular Vection) in Virtual Reality.” In Proceedings of 8th Annual Workshop Presence 2005, 49–57. http://en.scientificcommons.org/20596230.
Riecke, Bernhard E, Jorg Schulte-Pelkum, Franck Caniard, and Heinrich H Bülthoff. 2005. “Towards Lean and Elegant Self-Motion Simulation in Virtual Reality.” In Proceedings of the 2005 IEEE Conference 2005 on Virtual Reality, 131–38. VR ’05. https://doi.org/10.1109/VR.2005.83.
Riecke, B. E., D. Västfjäll, P. Larsson, and J. Schulte-Pelkum. 2005. “Top-Down and Multi-Modal Influences on Self-Motion Perception in Virtual Reality.” In Proceedings of HCI International 2005, 1–10. Las Vegas, NV, USA. http://en.scientificcommons.org/20596227.
Schulte-Pelkum, J., Bernhard E. Riecke, Franck Caniard, and Heinrich H. Bülthoff. 2005. “Can Auditory Cues Influence the Visually Induced Self-Motion Illusion?” Poster pre­sented at the European Conference on Visual Perception (ECVP).
Riecke, B. E., J. Schulte-Pelkum, M. N. Avraamides, and H. H Bülthoff. 2004. “Enhancing the Visually Induced Self-Motion Illusion (Vection) under Natural Viewing Conditions in Virtual Reality.” In Proceedings of 7th Annual Workshop Presence 2004, 125–32. https://doi.org/10.1.1.122.5636.
Riecke, B. E., J. Schulte-Pelkum, M. N Avraamides, M. von der Heyde, and Heinrich H Bülthoff. 2004. “Top-down Influence on Visually Induced Self-Motion Perception (Vection).” Poster pre­sented at the 7. Tübingen Perception Conference (TWK), Tübingen, Germany.
Riecke, B. E., J. Schulte-Pelkum, M. N. Avraamides, M. von der Heyde, and H. H. Bülthoff. 2004. “The Effect of Cognition on the Visually Induced Illusion of Self-Motion (Vection).” Journal of Vision 4 (8): 891a. https://doi.org/10.1167/4.8.891.
Schulte-Pelkum, J., B. E. Riecke, and H. H Bülthoff. 2004. “Vibrational Cues Enhance Believability of Ego-Motion Simulation.” Poster pre­sented at the International Multisensory Research Forum (IMRF), Barcelona, Spain.
Schulte-Pelkum, J., B. E. Riecke, M. von der Heyde, and H. H. Bülthoff. 2003. “Screen Curvature Does Influence the Perception of Visually Simulated Ego-Rotations.” Journal of Vision, Poster pre­sented at VSS, 3 (9). https://doi.org/doi: 10.1167/3.9.411.
Schulte-Pelkum, J., B. E. Riecke, M. von der Heyde, and H. H. Bülthoff. 2003. “Circular Vection Is Facilitated by a Consistent Photorealistic Scene.” Talk pre­sented at the Presence 2003 Conference, Aalborg, Denmark.