5.1. Introduction

Note

This report will focus on the development approach as well as evaluating the project and suggesting future improvements. For implementation details and how to use this project see the online documentation, where the gallery can also be found [1.1].

5.1.1. Purpose

The purpose of this thesis is to produce an extensible virtual reality (VR) viewer and editor for use with the libigl library [1.2]. In effect converting the current 2D user interface to the VR setting. Potential use cases include visualizations of 3D models and operations on them, such as those provided by libigl.

VR provides an alternative input and output format in comparison to a conventional 2D screen with a keyboard and mouse. It allows for accurate representation of 3D scenes, notably in terms of depth and scale, due to its stereoscopic rendering.

In terms of input, VR controllers can give precise 3D positional and rotational input for each hand in comparison to 2D positional input from a mouse. This is also superior to 3D mice, which only offer relative 3D positional and rotational input. VR is useful in our scenario as it allows for easier and more intuitive interaction with a mesh.

5.1.2. Related Work

Advances in VR input methods, with 6DOF controllers for each hand, has allowed for more innovation in 3D modeling. Several 3D modeling applications already exist on the Oculus Store such as Google Blocks [1.3] and Facebook’s Quill [1.4]. These offer construction of 3D scenes from primitives and brush strokes. Both use a ‘palette’-like user interface (UI) on the secondary hand for adding and manipulating meshes. Blocks has more basic functionality, but integrates with a sharing service online. Quill also offers snapping tools and being able to animate scenes.

Oculus Medium [1.5] provides more advanced functionality with sculpting and uses an unconventional method for storing its meshes allowing for fast boolean operations. “Medium defines 3D objects (sculpts) using an implicit surface. The surface is stored as a signed distance field (SDF) in a 3D grid of voxels.” [1.6].

Blender [1.7] has recently added a VR scene inspection [1.8] add-on, see also release notes [1.9]. This currently only enables the user to view a scene in VR and does not make use of the controllers. Unreal Engine [1.10] since version 4.17 has developed a VR mode [1.11] for its editor. This is primarily used for level design and previewing scenes. It replicates its 2D editor windows in VR as floating panels allowing a similar level of functionality to the 2D editor. It also makes use of nested pie menus for common actions like snapping. Both Blender and Unreal Engine allow for easy switching between 2D and VR.

In summary, there already exist several VR editors for 3D modeling and animation. However, this is still a developing field with little standardization. User interface is generally interacted with by using raycasting and a secondary hand is often dedicated for quick UI actions. Except for sculpting, complex deformation of meshes in VR has not yet been explored. This is what this thesis focuses on.

5.1.3. System Components

The Oculus Rift S headset was used as the primary target device.

For implementing this the Unity [1.12] game engine was chosen, partly due to experience with the engine. It provides many standard features as well as a cross-platform VR integration. It offers advanced VR features such as single-pass stereo rendering, which provides great performance benefits. Furthermore, it has an easy way of adding functionality via C# scripts. This, however, creates a necessary language interface to C++ such that libigl can be used.

Using the Oculus SDK directly requires too much development overhead and will result in less features. It will also be significantly harder to maintain. Using a game engine which already provides a range of features is the best option given the time available.

For example use cases of libigl two mesh deformations where chosen, a biharmonic deformation [1.13] and an As-Rigid-As-Possible deformation [1.14]. These each require selection of parts of the mesh as well as ways of transforming these, such that we can provide boundary conditions for the libigl algorithms. The deformations have been chosen from the libigl tutorial. The intent is that further libigl functionality, of any kind, can also be added.

Ideally, existing libigl applications would be able to simply switch which viewer is being used, either the current libigl 2D GLFW viewer and the VR viewer. This is not possible with this development approach. This is because libigl will be a library used by the VR editor and not the converse, due to how Unity executables are built. As a result, the interface to the VR viewer cannot be the same as the 2D GLFW viewer.

A workaround to this would be to implement inter-process communication between a libigl executable and a Unity built executable. This is however more involved and outside of the scope of this thesis. It is also unclear whether this approach will yield performant and maintainable results.

1.1

Roger Barton. 3D Modeling in Virtual Reality Documentation. https://vr-modeling.readthedocs.io/.

1.2

Alec Jacobson, Daniele Panozzo, and others. libigl: a simple C++ geometry processing library. 2018. https://libigl.github.io/.

1.3

Google. Google blocks. https://arvr.google.com/blocks/.

1.4

Facebook Technologies. Quill. https://quill.fb.com/.

1.5

Oculus. Oculus medium. https://www.oculus.com/medium/.

1.6

David Farrell. Medium under the hood: part 1 - developing the move tool (oculus developer blog). https://developer.oculus.com/blog/medium-under-the-hood-part-1-developing-the-move-tool.

1.7

Blender Foundation. Blender. https://www.blender.org/.

1.8

Blender Foundation. Blender VR scene inspection. https://docs.blender.org/manual/en/dev/addons/3d_view/vr_scene_inspection.html.

1.9

Blender Foundation. Blender VR scene inspection (release notes). https://wiki.blender.org/wiki/Reference/Release_Notes/2.83/Virtual_Reality.

1.10

Epic Games. Unreal engine. https://www.unrealengine.com/.

1.11

Epic Games. Unreal engine VR mode documentation. https://docs.unrealengine.com/en-US/Engine/Editor/VR/index.html.

1.12

Unity Technologies. Unity. https://unity.com/.

1.13

O. Sorkine, D. Cohen-Or, Y. Lipman, M. Alexa, C. Rössl, and H.-P. Seidel. Laplacian surface editing. In Proceedings of the 2004 Eurographics/ACM SIGGRAPH Symposium on Geometry Processing, SGP ‘04, 175–184. New York, NY, USA, 2004. Association for Computing Machinery. URL: https://doi.org/10.1145/1057432.1057456, doi:10.1145/1057432.1057456.

1.14

Olga Sorkine and Marc Alexa. As-rigid-as-possible surface modeling. In Proceedings of the Fifth Eurographics Symposium on Geometry Processing, SGP ‘07, 109–116. Goslar, DEU, 2007. Eurographics Association.