Embodied Self-Motion Illusions in VR

profile

How can we pro­vide humans with a believ­able sen­sa­tion of being in and moving through computer-generated envi­ron­ments (like VR, com­puter games, or movies) with­out the need for costly and cum­ber­some motion plat­forms or large free-space walk­ing areas? That is, how can we “cheat intel­li­gently” by pro­vid­ing a com­pelling, embod­ied self-motion illu­sion (“vec­tion”) with­out the need for full phys­i­cal motion?

In a series of exper­i­ments, we inves­ti­gated the con­tri­bu­tion and inter­ac­tion of dif­fer­ent sen­sory modal­i­ties as well as higher-level, cog­ni­tive influences.

Below is a selec­tion of such work [more will come as the web­site evolves]

Spatialized Sound Enhances Biomechanically-Induced Self-Motion Illusion (Vection)

The use of vec­tion, the illu­sion of self-movement, has recently been explored as a novel way to immerse observers in medi­ated envi­ron­ments through illu­sory yet com­pelling self-motion with­out phys­i­cally moving. This pro­vides advan­tages over exist­ing sys­tems that employ costly, cum­ber­some, and poten­tially haz­ardous motion plat­forms, which are often sur­pris­ingly inad­e­quate to pro­vide life-like motion expe­ri­ences. This study inves­ti­gates whether spa­tial­ized sound rotat­ing around the sta­tion­ary, blind­folded lis­tener can facil­i­tate bio­me­chan­i­cal vec­tion, the illu­sion of self-rotation induced by step­ping along a rotat­ing floor plate. For the first time, inte­grat­ing simple audi­tory and bio­me­chan­i­cal cues for turn­ing in place evoked con­vinc­ing cir­cu­lar vec­tion. In an audi­tory base­line con­di­tion, par­tic­i­pants expe­ri­enced only spa­tial­ized audi­tory cues. In a purely bio­me­chan­i­cal con­di­tion, seated par­tic­i­pants stepped along side­ways on a rotat­ing plate while lis­ten­ing to mono mask­ing sounds. Scores of the bi-modal con­di­tion (bin­au­ral bio­me­chan­i­cal cues) exceeded the sum of both single cue con­di­tions, which may imply super-additive or syn­er­gis­tic effects.

YouTube Preview Image

Stepping along a rotat­ing cir­cu­lar tread­mill while being blind­folded can (after a few sec­onds) create a com­pelling & embod­ied illu­sion of rotat­ing in the oppo­site direc­tion[ERROR: No URL was passed to the generic video BBCode]. Studying such “biomechnically-induced cir­cu­lar vec­tion” can help us to  better under­stand human multi-modal infor­ma­tion pro­cess­ing and even­tu­ally help us to improve motion sim­u­la­tion in Virtual Reality.

 

YouTube Preview Image

First demo of our new cir­cu­lar tread­mill in the iSpace lab. Since then the setup has had some major updates.

 

Why not build an omni-directional tread­mill so people can walk in all direc­tion in the Virtual Environment? While recent progress has been amaz­ing, such omni-directional tread­mills are still huge, loud, expen­sive, and need many safety mea­sures (see video below for an exam­ple from the Cyberwalk project). In addi­tion, it seems that walking-in-place can only reli­ably induce cir­cu­lar vec­tion, but not translational/linear vec­tion. Hence, we decided to focus on using out under­stand­ing if human multi-modal cue inte­gra­tion to “cheat intel­li­gently” and aiming for a com­pelling embod­ied illu­sion of self-motion (i.e., “per­cep­tual realism/effectiveness”) instead of aiming for biomechanical/stimulus realism.

YouTube Preview Image

Media Gallery

Related Publications

Feuereissen, D. 2008. “VR: Getting the Reality Part Straight – Does Jitter and Suspension of the Human Body Increase Auditory Circular Vection?” Bachelor’s Thesis, Department of Computer Science in Media. http://www.kyb.mpg.de/publication.html?publ=5071.
Palmisano, S., Nakamura, S., Allison, R. S., & Riecke, B. E. (2020). The Stereoscopic Advantage for Vection Persists Despite Reversed Disparity. Attention, Perception, & Psychophysics, 82(4), 2098–2118. https://doi.org/10.3758/s13414-019–01886-2)
Keshavarz, B., Phillip-Muller, A. E., Hemmerich, W., Riecke, B. E., & Campos, J. J. (2018). The effect of visual motion stim­u­lus char­ac­ter­is­tics on vec­tion and visu­ally induced motion sick­ness. Displays, 58, 71–81. https://doi.org/10.1016/j.displa.2018.07.005
Palmisano, S., & Riecke, B. E. (2018). The search for instan­ta­neous vec­tion: An oscil­lat­ing visual prime reduces vec­tion onset latency. PLOS ONE, 13(5), 1–26. https://doi.org/10.1371/journal.pone.0195886
Hashemian, A. M., Kitson, A., Nquyen-Vo, T., Benko, H., Stuerzlinger, W., & Riecke, B. E. (2018). Investigating a Sparse Peripheral Display in a Head-Mounted Display for VR Locomotion. 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 571–572. https://doi.org/10.1109/VR.2018.8446345
Mursic, R. A., Riecke, B. E., Apthorp, D., & Palmisano, S. (2017). The Shepard-Risset Glissando: Music that Moves You. Experimental Brain Research, 235(10), 3111–3127. https://doi.org/10.1007/s00221-017‑5033-1
Riecke, B. E. (2016). Using Spatialized Sound to Enhance Self-Motion Perception in Virtual Environments and Beyond: Auditory and Multi-Modal Contributions. Canadian Acoustics, 33(3), 148–149.
Keshavarz, B., Riecke, B. E., Hettinger, L. J., & Campos, J. L. (2015). Vection and visu­ally induced motion sick­ness: How are they related? Frontiers in Psychology, 6(413), 1–11. https://doi.org/10.3389/fpsyg.2015.00413
Riecke, B. E., Feuereissen, D., Rieser, J. J., & McNamara, T. P. (2015). More than a Cool Illusion? Functional Significance of Self-Motion Illusion (Circular Vection) for Perspective Switches. Frontiers in Psychology, 6(1174), 1–13. https://doi.org/10.3389/fpsyg.2015.01174
Riecke, B. E., Freiberg, J., & Grechkin, T. Y. (2015). Can Walking Motions Improve Visually Induced Rotational Self-Motion Illusions in Virtual Reality? Journal of Vision, 15(2), 1–15. https://doi.org/10.1167/15.2.3
Riecke, B. E., & Jordan, J. D. (2015). Comparing the effec­tive­ness of dif­fer­ent dis­plays in enhanc­ing illu­sions of self-movement (vec­tion). Frontiers in Psychology, 6(713). https://doi.org/10.3389/fpsyg.2015.00713
Riecke, Bernhard E., and Jörg Schulte-Pelkum. 2015. “An Integrative Approach to Presence and Self-Motion Perception Research.” In Immersed in Media: Telepresence Theory, Measurement and Technology, edited by Frank Biocca, Jonathan Freeman, Wijnand IJsselsteijn, Matthew Lombard, and Rachel Jones Schaevitz, 187–235. Springer. doi: 10.1007/978–3-319–10190-3_9.
Lawson, B.D., and Bernhard E. Riecke. 2014. “The Perception of Body Motion.” In Handbook of Virtual Environments: Design, Implementation, and Applications, edited by Kelly S. Hale and Kay M. Stanney, 2nd ed., 163–95. Ch. 7. CRC Press.
Riecke, Bernhard E., Daniel Feuereissen, John J. Rieser, and Timothy P. McNamara. 2014. “Can Self-Motion Illusions (Circular Vection) Facilitate Spatial Updating?” Poster pre­sented at the Spatial Cognition 2014 Conference, Bremen, Germany. http://conference.spatial-cognition.de/SC2014/.
Riecke, B. E, Jacqueline D. Jordan, Mirjana Prpa, and Daniel Feuereissen. 2014. “Underlying Perceptual Issues in Virtual Reality Systems: Does Display Type Affect Self-Motion Perception?” Talk pre­sented at the 55th Annual Meeting of the Psychonomic Society (Psychonomics), Los Angeles, USA.
Seno, T., Palmisano, S., Riecke, B. E., & Nakamura, S. (2014). Walking with­out optic flow reduces sub­se­quent vec­tion. Experimental Brain Research, 233(1), 275–281. https://doi.org/doi:10.1007/s00221-014‑4109-4
Riecke, Bernhard E. 2013. “Creating a Moving Experience  with­out Moving the Observer: Perceptual Aspects & Display Factors in Immersive Virtual Reality (and Elsewhere).” Invited Talk pre­sented at the International Toronto Stereoscopic Film Conference, Toronto, Canada. https://youtu.be/98hq_a_nvso.
Riecke, Bernhard E., and Jörg Schulte-Pelkum. 2013. “Perceptual and Cognitive Factors for Self-Motion Simulation in Virtual Environments: How Can Self-Motion Illusions (‘Vection’) Be Utilized?” In Human Walking in Virtual Environments, edited by Frank Steinicke, Yon Visell, Jennifer Campos, and Anatole Lécuyer, 27–54. New York: Springer. doi: 10.1007/978–1-4419–8432-6_2.
Freiberg, Jacob, and Bernhard E. Riecke. 2013. “Its Your Turn: Enhancing Visually Induced Self Motion Illusions (‘Vection’) with Walking Motions in Virtual Reality.” Poster pre­sented at the 54th Annual Meeting of the Psychonomic Society (Psychonomics), Toronto, Canada.
Riecke, B. E., Sigurdarson, S., & Milne, A. P. (2012). Moving Through Virtual Reality Without Moving? Cognitive Processing, 13(1), 293–297. https://doi.org/10.1007/s10339-012‑0491-7
Riecke, B. E. 2011. “Compelling Self-Motion Through Virtual Environments Without Actual Self-Motion – Using Self-Motion Illusions (‘Vection’) to Improve User Experience in VR. In J. Kim (Ed.).” In Virtual Reality, edited by Jae-Jin Kim, 149–76. Ch. 8. London, UK: InTechOpen. https://www.intechopen.com/books/virtual-reality/compelling-self-motion-through-virtual-environments-without-actual-self-motion-using-self-motion-ill.
Riecke, B. E., Väljamäe, A., & Schulte-Pelkum, J. (2009). Moving sounds enhance the visually-induced self-motion illu­sion (cir­cu­lar vec­tion) in vir­tual real­ity. ACM Transactions on Applied Perception (TAP), 6, 7:1–7:27. https://doi.org/10.1145/1498700.1498701
Riecke, B. E. (2009). Cognitive and higher-level con­tri­bu­tions to illu­sory self-motion per­cep­tion (“vec­tion”): does the pos­si­bil­ity of actual motion affect vec­tion? Japanese Journal of Psychonomic Science, 28(1), 135–139. http://ci.nii.ac.jp/naid/110007482465
Riecke, B. E., D. Feuereissen, and J. J. Rieser. 2009. “Rotating Sound Fields Can Facilitate Biomechanical Self-Motion Illusion (‘cir­cu­lar Vection’).” Journal of Vision 9 (8): 714–714. https://doi.org/10.1167/9.8.714.
Riecke, B. E., Feuereissen, D., & Rieser, J. J. (2009). Auditory self-motion sim­u­la­tion is facil­i­tated by haptic and vibra­tional cues sug­gest­ing the pos­si­bil­ity of actual motion. ACM Transactions on Applied Perception (TAP), 6, 20:1–20:22. https://doi.org/10.1145/1577755.1577763
“Contribution and Interaction of Auditory and Biomechanical Cues for Self-Motion Illusions (‘cir­cu­lar Vection’).” 2008. Poster pre­sented at the CyberWalk work­shop, Tübingen, Germany.
Riecke, B. E., Cunningham, D. W., & Bülthoff, H. H. (2007). Spatial updat­ing in vir­tual real­ity: the suf­fi­ciency of visual infor­ma­tion. Psychological Research, 71(3), 298–313. https://doi.org/10.1007/s00426-006‑0085-z
Schulte-Pelkum, J. 2007. “Perception of Self-Motion: Vection Experiments in Multi-Sensory Virtual Environments.” PhD thesis, Ruhr-Universität Bochum. https://hss-opus.ub.ruhr-uni-bochum.de/opus4/frontdoor/deliver/index/docId/2735/file/diss.pdf.
Riecke, B. E., Schulte-Pelkum, J., Avraamides, M. N., Heyde, M. V. D., & Bülthoff, H. H. (2006). Cognitive fac­tors can influ­ence self-motion per­cep­tion (vec­tion) in vir­tual real­ity. ACM Transactions on Applied Perception (TAP), 3, 194–216. https://doi.org/10.1145/1166087.1166091
Riecke, B. E., & Schulte-Pelkum, J. (2006). Using the per­cep­tu­ally ori­ented approach to opti­mize spa­tial pres­ence & ego-motion sim­u­la­tion (153). MPI for Biological Cybernetics. http://www.kyb.mpg.de/publication.html?publ=4186
Riecke, B. E., J. Schulte-Pelkum, and F. Caniard. 2006. “Visually Induced Linear Vection Is Enhanced by Small Physical Accelerations.” Poster pre­sented at the 7th International Multisensory Research Forum (IMRF), Dublin, Ireland.
Riecke, B. E., Schulte-Pelkum, J., Caniard, F., & Bülthoff, H. H. (2005). Spatialized audi­tory cues enhance the visually-induced self-motion illu­sion (cir­cu­lar vec­tion) in Virtual Reality. (138). MPI for Biological Cybernetics. http://www.kyb.mpg.de/publication.html?publ=4187
Riecke, B. E., J. Schulte-Pelkum, F. Caniard, and Heinrich H. Bülthoff. 2005. “Auditory Cues Can Facilitate the Visually-Induced Self-Motion Illusion (Circular Vection) in Virtual Reality.” Poster pre­sented at the 8. Tübingen Perception Conference (TWK), Max Planck Institute for Biological Cybernetics, Germany.
Riecke, B. E., J. Schulte-Pelkum, F. Caniard, and H. H Bülthoff. 2005. “Influence of Auditory Cues on the Visually-Induced Self-Motion Illusion (Circular Vection) in Virtual Reality.” In Proceedings of 8th Annual Workshop Presence 2005, 49–57. http://en.scientificcommons.org/20596230.
Schulte-Pelkum, J., Bernhard E. Riecke, Franck Caniard, and Heinrich H. Bülthoff. 2005. “Can Auditory Cues Influence the Visually Induced Self-Motion Illusion?” Poster pre­sented at the European Conference on Visual Perception (ECVP).
Riecke, B. E., J. Schulte-Pelkum, M. N Avraamides, M. von der Heyde, and Heinrich H Bülthoff. 2004. “Top-down Influence on Visually Induced Self-Motion Perception (Vection).” Poster pre­sented at the 7. Tübingen Perception Conference (TWK), Tübingen, Germany.
Riecke, B. E., J. Schulte-Pelkum, M. N. Avraamides, M. von der Heyde, and H. H. Bülthoff. 2004. “The Effect of Cognition on the Visually Induced Illusion of Self-Motion (Vection).” Journal of Vision 4 (8): 891a. https://doi.org/10.1167/4.8.891.
Schulte-Pelkum, J., B. E. Riecke, and H. H Bülthoff. 2004. “Vibrational Cues Enhance Believability of Ego-Motion Simulation.” Poster pre­sented at the International Multisensory Research Forum (IMRF), Barcelona, Spain.
Schulte-Pelkum, J., B. E. Riecke, M. von der Heyde, and H. H. Bülthoff. 2003. “Screen Curvature Does Influence the Perception of Visually Simulated Ego-Rotations.” Journal of Vision, Poster pre­sented at VSS, 3 (9). https://doi.org/doi: 10.1167/3.9.411.
Schulte-Pelkum, J., B. E. Riecke, M. von der Heyde, and H. H. Bülthoff. 2003. “Circular Vection Is Facilitated by a Consistent Photorealistic Scene.” Talk pre­sented at the Presence 2003 Conference, Aalborg, Denmark.