Siggraph 2018: 3D User Interfaces for Virtual Reality and Games: 3D Selection, Manipulation, and Spatial Navigation

3-hour Course Presented at Siggraph 2018

In this course, we will take a detailed look at dif­fer­ent topics in the field of 3D user inter­faces (3DUIs) for Virtual Reality and Gaming. With the advent of Augmented and Virtual Reality in numer­ous appli­ca­tion areas, the need and inter­est in more effec­tive inter­faces becomes preva­lent, among others driven for­ward by improved tech­nolo­gies, increas­ing appli­ca­tion com­plex­ity and user expe­ri­ence require­ments. Within this course, we high­light key issues in the design of diverse 3DUIs by look­ing closely into both simple and advanced 3D selection/manipulation and spa­tial nav­i­ga­tion inter­face design topics. These topics are highly rel­e­vant, as they form the basis for most 3DUI-driven appli­ca­tion, yet also can cause major issues (per­for­mance, usabil­ity, expe­ri­ence. motion sick­ness) when not designed prop­erly as they can be dif­fi­cult to handle. Within this course, we build on top of a gen­eral under­stand­ing of 3DUIs to dis­cuss typ­i­cal pit­falls by look­ing closely at the­o­ret­i­cal and prac­ti­cal aspects of selec­tion, manip­u­la­tion, and nav­i­ga­tion and high­light guide­lines for their use.

Intended Audience

Developers, researchers, psy­chol­o­gists, and user-experience pro­fes­sion­als who want to adopt new ways of inter­act­ing in Virtual Reality using state of the art 3D User inter­face design strate­gies, rang­ing from 3D selec­tion and manip­u­la­tion to spa­tial navigation.

Audience Takeaways

By par­tic­i­pat­ing in two con­sec­u­tive and log­i­cally inter­linked ses­sions, par­tic­i­pants will acquire nec­es­sary skill set to design, develop and val­i­date 3D inter­faces and tech­niques for vir­tual real­ity and gaming sys­tems. In sum­mary, the ses­sions will enable par­tic­i­pants to acquire the nec­es­sary knowl­edge and skills through cov­er­ing spa­tial nav­i­ga­tion and 3D selection/manipulation topics, grouped in the two blocks of the course.

With the increas­ing avail­abil­ity, qual­ity, and afford­abil­ity of immer­sive gaming and virtual/augmented/mixed real­ity hard– and soft­ware, there is an increas­ing need for improved inter­faces that com­bine high usabil­ity and learn­abil­ity with a more embod­ied inter­ac­tion that goes beyond gamepads and mouse/keyboard approaches. This topic is highly rel­e­vant, as many appli­ca­tion domains ben­e­fit from well-performing tech­niques, from games to 3D design exploration/review and big data explo­ration sce­nar­ios. Furthermore, due to the recent (re)advent of vir­tual real­ity, this area has become even more rel­e­vant as many users are affected by the (lack of) per­for­mance of 3D inter­faces and tech­niques. Hence, we will cover the full range from hand-operated to full-body con­trolled inter­faces, from low-cost to high-end.

Course structure

Both the selec­tion and manip­u­la­tion and the spa­tial nav­i­ga­tion blocks deal with foun­da­tions and prac­ti­cal aspects to create a solid foun­da­tion for cre­at­ing 3D user inter­faces. While other 3D user inter­face tasks, such as system con­trol, exist, nav­i­ga­tion and selection/manipulation form the pre­dom­i­nant tasks in most 3D user inter­faces, and thus are highly representative.

Both the 3D Selection and Manipulation and the spa­tial nav­i­ga­tion blocks will start by cov­er­ing the fun­da­men­tals and applic­a­ble usage domains. This includes a clas­si­fi­ca­tion of dif­fer­ent inter­faces and tech­niques (with pros and cons) as well as other issues includ­ing adverse effects such as motion sick­ness and usabil­ity issues will be dis­cussed, along with a set of guide­lines to alle­vi­ate them.

In the sub­se­quent more applied prac­tice sec­tion of each block, we will pro­vide an overview of the method­olo­gies to design, develop, and val­i­date novel inter­faces and inter­ac­tion tech­niques, again accom­pa­nied with a set of guide­lines. Attendees will also be guided through var­i­ous real-world design exam­ples that high­light the foun­da­tions and prac­ti­cal design deci­sions, devel­op­ment issues, and val­i­da­tion methodologies.

 

Speaker Biographies

Bernhard Riecke is asso­ciate pro­fes­sor in the School of Interactive Arts & Technology (SIAT) at Simon Fraser University. Riecke received his PhD from Tübingen University in 2003 and worked for a decade in the Virtual Reality group (Cyberneum) at the Max Planck Institute for Biological Cybernetics, as well as Vanderbilt and UC Santa Barbara. His work spans the­o­ret­i­cal and applied domains and is pub­lished in jour­nals includ­ing Frontiers, JOV, ACM-TAP, Cognition, and Presence, and con­fer­ences includ­ing IEEE VR, ACM-CHI, ACM SIGGRAPH, ACM-SUI, ISEA, and Spatial Cognition. Bernhard recently gave a TEDx talk on “Could Virtual Reality make us more Human”.
Joseph J. LaViola Jr. is an asso­ciate pro­fes­sor in the Department of Computer Science at the University of Central Florida. He directs the Interactive Computing Experiences Research Cluster of Excellence and is also an adjunct asso­ciate research pro­fes­sor in the Computer Science Department at Brown University. He is the lead author on the second edi­tion of “3D User Interfaces: Theory and Practice”, the first com­pre­hen­sive book on 3D user inter­faces. Joseph received a Sc.M. in Computer Science in 2000, a Sc.M. in Applied Mathematics in 2001, and a Ph.D. in Computer Science in 2005 from Brown University.
Ernst Kruijff is interim pro­fes­sor at the Institute of Visual Computing, Bonn-Rhein-Sieg University of applied sci­ences and adjunct pro­fes­sor at Simon Fraser University. His work has been pre­sented at con­fer­ences such as IEEE VR, 3DUI and ISMAR, and ACM VRST. Ernst has pre­sented numer­ous courses about 3DUI topics, includ­ing ACM SIGGRAPH, CHI and VRST, and IEEE VR. Ernst is co-author of the stan­dard ref­er­ence in the field of 3DUIs (LaViola et al. 3D User Interfaces: Theory and Practice, 2017).

 

Supplementary materials

Some of the course con­cepts are based on the pre­sen­ters’ book: Joseph J. LaViola Jr, Ernst Kruijff, Ryan P. McMahan, Doug Bowman, and Ivan P. Poupyrev. 2017. 3D User Interfaces: Theory and Practice (2^nd ed.). Addison-Wesley.

Draft course notes and slides can be down­loaded here. please cite as: Riecke, B. E., LaViola Jr., J. J., & Kruijff, E. (2018). 3D User Interfaces for Virtual Reality and Games: 3D Selection, Manipulation, and Spatial Navigation. In Proceedings of ACM SIGGRAPH 2018 Courses (SIGGRAPH ’18). (half-day course). Vancouver, BC, Canada

Videos for the nav­i­ga­tion parts are directly linked in the .pdf slides. Videos of the 3D selec­tion and manip­u­la­tion parts are listed here: