Project Ideas for CS 8803 3DU

Project Ideas

To help with generating appropriate project ideas, I’ve put together this page to help you come up with ideas. It has some suggestions, but mostly points to places to look for inspiration, and how to think about the project. We’ll create a channel in Teams to discuss more ideas.

Some of these ideas were taken from a similar page created for the 2D UI Software Class (CS6456) in the past, some of which are taken from research papers, some of which are ideas I or others have come up with. Please don’t consider yourself limited to just these ideas–these are just a starting point, you’re encouraged to come up with cool 3D UI techniques on your own. This list is just meant to convey the scope and scale of what we’re looking for. Also note that short descriptions like this merely convey the topic area of possible projects, they don’t describe all of the details about what would be sufficient to get a good grade on the project. (In other words, for any of these it’s possible to do a very bad version or a very good version–simply choosing a topic on this list is insufficient to guarantee any particular grade.)

In general, good project ideas will represent a solid chunk of implementation work (at least equivalent to three homework assignments, so think at about this scale). You might considering looking at the 3D, AR, and VR interaction technique papers at the last few years of the ACM Symposium on Virtual Reality Software and Technology (VRST), the IEEE VR and 3D User Interface conferences, ACM User Interface Software and Technology conference, as well as relevant papers in conferences like CHI, Web3D, and SIGGRAPH. Focus on papers that present interesting and novel 3D interaction techniques, not applications or uses of 3D technologies.

You project will hopefully represent a novel interaction technique, or at least a novel application of an existing interaction technique that solves an important or interesting problem. For example, just reimplementing a 3D menu system from some 3D application you have seen is not very novel, since it’s likely been done many times. Beyond that, a straightforward 3D menu system are probably of insufficient complexity to make a good project (they’re probably less than one homework’s worth of effort). However, general 3D menuing is an unsolved problem, given the problems we discuss in class with respect to haptics, affordances, unreliability, latency, and so on. Building on previous work and a reasoned analysis of the problems and opportunities with current devices, and developing a system that is demonstrably better that existing techniques (in a way that you can explain and justify) would be a good project.

That said, in addition to looking at the conferences, here are some projects ideas.

• 3D sketch beautification. There has been a lot of work on creating 3D models from 2D sketches. For example, see Igarashi’s Pegasus system, or Sketch/SmoothSketch/etc from Hughes and Zeleznik at Brown. Apply similar techniques to free-form drawing in VR, or use them more directly for AR on phones and tablets. You would need to explore commercial VR paint and design applications to understand the state of the art in industry, as well as academia.
• 3D stroke and shape beautification. Instead of creating 3D models, focus on cleaning up freeform sketching. For example, see Adobe’s “Comp CC” tool on the App Store, which uses hand-drawn shapes to create layouts, or Landay’s SILK tool for sketching UI layouts that can actually be run. Imagine such systems in 3D.
• UI tools supporting progressive refinement across diverse platforms. Toolkits like Microsoft’s Mixed Reality Toolkit (for Unity) provide some common 3D UI metaphors and capabilities, but are focused on high end AR/VR platforms. WebXR supports a wide range of platforms, from flat screens to AR/VR on phones to 3DOF and 6DOF AR and VR. Design and build a set of tools that provide a basic collection of UI metaphors for the full range of platforms, within a Babylon.js or three.js.
• Taking inspiration from the available tools in the MR Toolkit for Unity, and comparing to what is available for Babylon or three.js, create a set of unique and interesting UI tools for WebXR.
• Design and develop 3D UI’s based on hand tracking, to solve some class of problems. There has been academic work in the past, but has been a lot more exploration lately based on Leap Motion (now “Ultraleap”), and will soon be more thanks to the new Oculus and Hololens2 hand tracking. See UltraLeap’s stream for random ideas.
• Create a hybrid 2D/3D UI toolkit for use in hybrid 2D/AR apps on phones. Putting standard platform 2D content into 3D is difficult, both using the web and on native. three-web-layer shows how to get (slowly-updated) web 2D content into 3D, but does not address how to transform those interfaces substantially to leverage 3D UI techniques. Create a toolkit that lets you create full-featured 2D interfaces that have different-but-appropriate versions in 2D and 3D, when the user moves between 2D and AR views on their phone.
• Develop a set of interaction techniques for handheld AR based on multitouch gesture. Build on both the multitouch UI research and toolkits, and the multi-finger/multi-hand work in 3D UIs, taking advantage of the stable multi-touch and spatial tracking provided by ARKit/ARCore/WebXR on modern phones.
• Tools for immersive authoring and editing in VR. Survey start of the art in current applications, plus research literature. Get inspiration from vides like WorldBuilder. Identify opportunities for a unique and interesting authoring toolkit.
• Tools for immersive authoring and editing in AR. There are similarities and differences between AR and VR, related both to navigation/locomotion, but also to the opportunity to leverage sensing of the physical world, and the uncertainty/noisiness of the sensing. Research going back to Tinmith has looked at outdoor authoring.
• Create Magic Lenses for immersive debugging of Babylon or three.js scenes. See the original Magic Lens paper and Hudson’s Debugging Lenses paper for inspiration.
• CAD systems rely on multiple simultaneous orthographic views for reliable editing. Explore how to use multiple viewpoints in an immersive drawing/editing application.
• Develop a set of interfaces for collaborative organization and manipulation of content, inspired by research systems systems in 3D but also tabletop and desktop interfaces that use 3D metaphors (like Bumptop, Task Gallery, or Michel Beaudouin-Lafon’s Novel Interaction Techniques for Overlapping Windows and videos) and commercial systems such as Spatial.is.