When is Virtual Reality real enough? Using VR in spatial cognition research

Abstract

Although the use of Virtual Reality in spa­tial cog­ni­tion research is becom­ing more and more wide­spread, there is little guid­ance on what kind of VR setups and exper­i­men­tal par­a­digms are most suit­able for spe­cific sit­u­a­tions and research ques­tions. This might result in a number of issues, such as sub­op­ti­mal use of equip­ment and insuf­fi­cient trans­fer­abil­ity of results to the real world.

The aim of this half-day work­shop is to dis­cuss and raise aware­ness of these issues, and take first steps in tack­ling them. The work­shop will start with a brief intro­duc­tion of the cur­rent state of VR in spa­tial cog­ni­tion research, fol­lowed by indi­vid­ual talks on spe­cific research projects in this area. The work­shop will con­clude with a gen­eral panel discussion.

Attendees will leave with an improved under­stand­ing of the advan­tages and chal­lenges of using var­i­ous VR setups to address dif­fer­ent research ques­tions. No prior VR knowl­edge or expe­ri­ence is required for attendees.

Important dates:

Workshop date: August 31, 2012, from 8:30–12:00, as part of the Spatial Cognition 2012 conference

Organizers:

Bernhard Riecke (Simon Fraser University)

Introductory short pre­sen­ta­tion: “When is Virtual Reality Real Enough? Embodied vs. Disembodied Spatial Cognition in VR”

 

 

Klaus Gramann (Technical University Berlin)

Introductory short pre­sen­ta­tion: “Disembodiment in VR Spatial Cognition Research: a Neuroscience Perspective”

 

 

Presenters:

Frank Steinicke (University of Würzburg)

“Making Small Spaces Feel Large: Self-Motion Perception, Redirection and Illusions”

 

 

 

Maria Kozhevnikov (Harvard Medical School, University of Singapore)

“Understanding immer­siv­ity from cog­ni­tive neru­o­science perspective”

 

 

 

 

Christian Doeller (Radboud University Nijmegen)

“Virtual real­ity in neu­ro­science: bridg­ing between cells, sys­tems and behav­iour”

 

 

 

Nicholas Giudice (University of Maine)

“Moving beyond the vision-only bot­tle­neck: Using mul­ti­modal infor­ma­tion in vir­tual real­ity research“

 

 

 

 

---

Tentative Timeline

8:30 - 10:00, Part 1: Presentations

8:30: Bernhard Riecke (Simon Fraser University)
Introductory short pre­sen­ta­tion: “When is Virtual Reality Real Enough? Embodied vs. Disembodied Spatial Cognition in VR”

Propose gen­eral guidelines:

• as we have lim­ited time we’ll need to reduce ques­tions during talks — instead, please jot down your thoughts&questions you’d like to dis­cuss in during second part of the workshop
• • we’ll hand out post-it notes, and invite you to put them on the wall before/during coffee break, so we can start gath­er­ing, cat­e­go­riz­ing & pri­or­i­tiz­ing poten­tial dis­cus­sion questions
• use coffee break to start brain­storm­ing & gather around dis­cus­sion topics etc.

8:45: Klaus Gramann (Technical University Berlin)

Introductory short pre­sen­ta­tion: “Disembodiment in VR Spatial Cognition Research: a Neuroscience Perspective”

8:55 Frank Steinicke (University of Würzburg)
“Making Small Spaces Feel Large: Self-Motion Perception, Redirection and Illusions”

9:10 Maria Kozhevnikov (Harvard Medical School, University of Singapore)
“Understanding immer­siv­ity from cog­ni­tive neru­o­science perspective”

9:25 Christian Doeller (Radboud University Nijmegen)
“Virtual real­ity in neu­ro­science: bridg­ing between cells, sys­tems and behav­iour”

9:40 Nicholas Giudice (University of Maine)
“Moving beyond the vision-only bot­tle­neck: Using mul­ti­modal infor­ma­tion in vir­tual real­ity research“

9:55: wrap-up & out­look of ses­sion 2 (Bernhard)

10:00-10:30 coffee break

Suggestion: use coffee break to put up your post-it notes with dis­cus­sion ques­tions & start brain­storm­ing & gath­er­ing around dis­cus­sion topics…

10:30-12:00, Part 2: Brainstorming, Breakout group discussion & Final Plenum Discussion

10:30 Overview of pos­si­ble dis­cus­sion ques­tions based on your post-it notes

10:35 Brainstorm more ques­tions, gather (e.g., Bernhard using MindMap soft­ware?), decide on most inter­est­ing dis­cus­sion topics (open questions).

10:40 break out into groups of about 5 people with one main focus question

• decide on note­taker (should have laptop or use flipchart) to sum­ma­rize essence of discussion.
• decide on group reporter (who will briefly present essence of discussion/insights to plenum: New insights? Suggestions? Tips&Tricks? New ques­tions? What stood out for you or was surprising/most exciting/stimulating/useful? How could we use the insights in the future? Ideas for future projects/collaborations?)
• make sure all team mem­bers contribute
• if inter­net access is avail­able: write directly into shared google doc, else use memory stick or pass net­work cable around…

11:30 break-out groups present results in plenum

11:50–12:00 Conclusions, out­look, poten­tial next steps

---

---

General Structure

Part 1: Introduction and Presentations

This half-day work­shop will be divided into two parts. First, Bernhard and Klaus will briefly intro­duce the over­all topic, pro­vide a bit of a back­ground on embod­ied vs. dis­em­bod­ies spa­tial cog­ni­tion in VR, and con­clude by rais­ing some ques­tions and cur­rent chal­lenges, thus fram­ing the upcom­ing pre­sen­ta­tions and dis­cus­sions. This is fol­lowed by 4 experts (see above) pre­sent­ing and dis­cussing their research and expe­ri­ence with using VR setups in the con­text of spa­tial cog­ni­tion research, as indi­cated above. Further dis­cus­sion topics might include:

• The role of phys­i­cal motion cues in spa­tial orientation
• Sensory con­flicts and simulator/motion sickness
• Different dis­play para­me­ters and their effects
• The use of self-motion illu­sions to “fake” phys­i­cal motion
• Different strate­gies for and under­ly­ing neural rep­re­sen­ta­tions of spa­tial ori­en­ta­tion in VR

Part 2: Interactive Brainstorming and Discussion Workshop

The second half of the ses­sion will be more inter­ac­tive and workshop-style: After a short brain­storm­ing ses­sion, we will col­lect and orga­nize the most inter­est­ing and rel­e­vant dis­cus­sion topics, and will spend the remain­ing time dis­cussing these in smaller groups (depend­ing on the over­all work­shop size). Topics might include 

• the cur­rent state and future of VR in spa­tial cog­ni­tion research
• expe­ri­ences, strate­gies and chal­lenges in using VR as a research tool
• guide­lines for its opti­mal usage.

One or more note­tak­ers will directly doc­u­ment these dis­cus­sions, and we will put the out­come of the work­shop online soon after­wards (at least for par­tic­i­pants). If there’s suf­fi­cient inter­est, energy, and excite­ment, we hope this might kick­start fur­ther dis­cus­sion and collaboration.

Organization and goal:

Virtual Reality can be a highly useful research plat­form in spa­tial cog­ni­tion. It pro­vides a unique oppor­tu­nity to study human per­cep­tion and behav­ior in immer­sive, multi-modal and nat­u­ral­is­tic vir­tual envi­ron­ments under repro­ducible and con­trol­lable exper­i­men­tal conditions.

For an exam­ple, imag­ine a researcher study­ing human nav­i­ga­tion in city envi­ron­ments. A typ­i­cal exper­i­ment in this field might mea­sure par­tic­i­pants’ ori­en­ta­tion and sense of direc­tion as they nav­i­gate in an actual city. This presents some chal­lenges. For one, the researcher must select an exist­ing envi­ron­ment to use. As the selec­tion pool is usu­ally lim­ited by geo­graph­i­cal loca­tion, the researcher might have to settle for an envi­ron­ment that is less suit­able than what she had in mind. Also, she must relin­quish con­trol of sev­eral pos­si­ble con­founds that can affect the valid­ity of her study. These include prior knowl­edge, weather, vis­i­bil­ity, sound, ori­ent­ing cues like the direc­tion of sun­light, wind, or global land­marks, and other inter­fer­ence or con­founds from the environment.

Under the right cir­cum­stances VR pro­vides an ele­gant solu­tion to these prob­lems. Rather than set­tling for an exist­ing envi­ron­ment, a vir­tual city can be built accord­ing to exact spec­i­fi­ca­tions. Environments can even be adjusted and changed on-the-fly as needed. Confounding vari­ables can be sig­nif­i­cantly reduced, as more fac­tors can be directly con­trolled and mea­sured. Finally, depend­ing on the sit­u­a­tion, a VR-based study can be safer and more cost-effective than a cor­re­spond­ing study per­formed in the real world, such as research into spa­tial cog­ni­tion during flight or poten­tially dan­ger­ous sit­u­a­tions like evac­u­a­tion scenarios.

However, the use­ful­ness and util­ity of VR in research crit­i­cally depends on the spe­cific cir­cum­stances. That is, it can be ben­e­fi­cial or detri­men­tal to study valid­ity and real-world trans­fer­abil­ity, depend­ing on the task, envi­ron­ment or research focus. Correctly eval­u­at­ing under what con­di­tions which kind of VR setup might be most suit­able can be chal­leng­ing, espe­cially as tech­nol­ogy advances. It is often simply assumed that a give VR setup is suit­able and pro­vides real-world trans­fer­abil­ity, simply because it is a costly and state-of-the-art setup. This is, how­ever, not nec­es­sar­ily the case, as will be dis­cussed in the workshop.

Our goal with this work­shop is to raise aware­ness for these issues, share and dis­cuss rel­e­vant find­ings and expe­ri­ences, and thus enable researchers to make more informed deci­sions regard­ing the usage and util­ity of dif­fer­ent types of VR setups and inter­ac­tion par­a­digms in their research. Participants will leave with a better under­stand­ing of when they can use VR to their advan­tage and how they can go about imple­ment­ing it. In addi­tion, we hope to strengthen a net­work of researchers in this area that will extend beyond the time of the workshop.

Potential Participants:

This work­shop will be most ben­e­fi­cial to researchers who actively design and carry out stud­ies in spa­tial cog­ni­tion and cur­rently use or plan on using VR tech­nol­ogy to do so. This includes stud­ies of spa­tial nav­i­ga­tion, ori­en­ta­tion, memory, learn­ing, wayfind­ing, and other fields that rely on exper­i­men­tal paradigms.