IAT 848 Mediated, Virtual, and Augmented Reality

This course is the Fall 2024 ver­sion of IAT 848 Mediated, Virtual, and Augmented Reality, taught by Dr. Bernhard Riecke at Simon Fraser University.

Upcoming Showcase: Tuesday November 26th, from 2:30 - 5:30pm, in SRYC 3850 and 3800 (aka black box and gray box labs).

Project web­sites:

MindScape VR: Promoting brain health

..If I Were You..

BR - Fly: Become a firefly through a guided breathing exercise

Course Details

Calendar Description:

Covers the emerg­ing field of vir­tual, aug­mented, medi­ated, and mixed real­ity from human-centered, research, tech­ni­cal, and eth­i­cal per­spec­tives. Discusses and ana­lyzes design, devel­op­ment, usage, and eval­u­a­tion of tech­nolo­gies that can be used to medi­ate human expe­ri­ence and inter­ac­tion with vir­tual and real envi­ron­ments includ­ing Virtual, Augmented, and Mixed Realities (together known as XR). Investigates how these emerg­ing tech­nolo­gies can affect and aug­ment human per­cep­tual, motor, cog­ni­tive and socio-emotional processes. Analyzes human-centered approaches to inter­ac­tion with 3D real and vir­tual con­tent, using visual, audi­tory, haptic, kines­thetic, phys­i­o­log­i­cal and neu­ro­phys­i­o­log­i­cal modal­i­ties. Design guide­lines and prac­tices are cov­ered through­out. Considers aes­thetic, cul­tural and eth­i­cal impli­ca­tions of medi­at­ing reality.

Class location and times

Each week, start­ing Tuesday Sept 10th, we’ll be for the first 2 hours in the com­puter lab (that has VR capa­ble com­put­ers) in case we need them, and then will move to another nearby sem­i­nar room for the last hour (as the com­puter lab wasn’t avail­able for the whole 3 hours unfortunately):

Tuesday, 2:30pm-4:20pm, SRYC 3050

Tuesday, 4:30pm-5:20pm, SRYC 3120 (both on the SFU Surrey campus, main Surrey Central build­ing, Galleria 3)

Course-level educational goals and desired learning outcomes

After successfully completing the course, students should be able to do the following 

  • Critically engage with, reflect, dis­cuss, and ana­lyze inter­ac­tive VR/MR/AR (abbre­vi­ated as XR) expe­ri­ences using and apply­ing rel­e­vant schol­arly frame­works, the­o­ries, and concepts
  • Explain, eval­u­ate, dis­cuss cur­rent chal­lenges of XR on the tech­ni­cal, eth­i­cal, per­cep­tual, and user expe­ri­ence level
  • Prepare a XR research pro­posal and eval­u­ate its fea­si­bil­ity, includ­ing a clear moti­va­tion and argu­ment for the gap in lit­er­a­ture and cur­rent state of the art and and argu­ment for its contribution
  • Design and create a real-time immersive/XR expe­ri­ence that iden­ti­fies the con­text of par­tic­i­pants and stake­hold­ers, and takes advan­tage of the poten­tial of the tech­nol­ogy. This includes being able to argue con­vinc­ingly why it makes sense to use the chosen tech­nol­ogy and its poten­tial inte­gra­tion in larger systems.
  • Being able to design, run, ana­lyze and present user studies/evaluation/research of XR system/user experience/performance

Requirements / prerequisite

  • Basic pro­gram­ming skills, as doc­u­mented through IAT806 or equiv­a­lent demon­stra­tion or doc­u­men­ta­tion of basic pro­gram­ming com­pe­tency, or instruc­tor permission.
  • Basic research meth­ods skills, as doc­u­mented through having taking a grad­u­ate research meth­ods course (e.g., IAT 801, 802, 803, 834), or instruc­tor permission.

Teaching/learning activities include:

  • Interactive lec­tur­ing and demon­stra­tions, includ­ing flipped-classroom com­po­nents (read­ing and video tuto­ri­als at home incen­tivized by assignments)
  • Group dis­cus­sions (in-class or online chat– and dis­cussing forums)
  • Regular read­ing, writ­ing and/or revision/reviewing/feedback assignments
  • Online and in-class tuto­ri­als on var­i­ous topics rang­ing from con­cep­tual to technical
  • Group/individual research projects and presentations

Topics and Overview

As this field is rapidly evolv­ing topics cov­ered will evolve over time. Current topics include but are not lim­ited to:

  • Introduction into medi­ated, vir­tual and aug­mented realities:
    • Overview, Background and Motivation
    • Definitions
    • History and development
    • Different kinds of alter­nate realities
    • Presence, immer­sion, and ref­er­ence frames
  • Overview of technologies
    • Hardware, soft­ware, interfaces,
    • How to move through alter­nate realities
    • How to inter­act with alter­nate realities
  • Designing for human capabilities:
    • Perceptual, cog­ni­tive, sen­so­r­ial & phys­i­o­log­i­cal modalities
    • The human in mul­ti­ple realities
  • What can go wrong: Adverse side effects
    • From motion sick­ness to dis­ori­en­ta­tion, strain, usabil­ity, and prac­ti­cal challenges
    • Design guide­lines
  • Closing the action-perception loop: Interaction and feedback
    • The human in the loop
    • Input, output, and what hap­pens in between
    • Interaction par­a­digms
  • How to design alter­nate real­i­ties and MR/VR/AR content
    • Principles for design­ing for alter­nate realities
    • Determining con­text: e.g., train­ing, learn­ing, explor­ing, gaming, storytelling/narratives, socially inter­act­ing (syn­chro­nously & asyn­chro­nously), visu­al­iz­ing data & sensemaking
    • Iterative design and evaluation
      • Co-creating with Participants, Stakeholders, Communities
      • Intertwining tech­nol­ogy, aes­thet­ics (look & feel) & interaction
      • Assessing con­tent & con­text: account­ing for use sce­nar­ios in spe­cific con­texts; siting: access, sustainability/longitudinal use, fit with insti­tu­tional, social con­texts, needs & expec­ta­tions; account­ing for lon­gi­tu­di­nal use: upgrades, tech­ni­cal sup­port; evo­lu­tion of use
  • Innovation, entre­pre­neur­ial considerations
  • User inter­face guide­lines for alter­nate realities
  • User stud­ies in alter­nate realities
  • Implications, poten­tials and chal­lenges in design­ing alter­nate realities
    • Scientific, health, and tech­no­log­i­cal aspects
    • What should or should we not design? Social, soci­etal, cul­tural, and eth­i­cal perspectives
    • Artistic, aes­thetic & nar­ra­tive perspectives
  • Responsibility, ethics, and the future of alter­nate realities

Current research topics and chal­lenges as well as design guide­lines and prac­tices will be cov­ered through­out the course and above topics.

Examples of prior students' VR/XR projects from Bernhard's classes:

Grad/undergrad mixed:

UBISS Summerschool 2024 1-week inten­sive VR Workshop/Jam “Virtually Real? The Art and Science of Designing Impactful (or Even Transformative?) Virtual Experiences”

Undergrad classes

Semester in Alternate Realities

IAT 445: Immersive Environments

IAT 499: Graduation Project

Resources

[excerpt, see Canvas course resources page for more details]

Unity & VR

if you don’t have a strong back­ground in pro­gram­ming or VR devel­op­ment already, I’d sug­gest to go through some basic Unity tuto­ri­als.  It’ll be useful to have at least a basic under­stand­ing of these areas for the course projects. Prior stu­dents have found the fol­low­ing tuto­ri­als quite useful:

Quantitative research methods

see my IAT 802 Quantitative Research Methods and Design tutorials/guidelines