Showing the Point: Understanding and Representing Deixis over Surfaces

Abstract

Deictic gestures, which often manifest as pointing, are an important part of interpersonal communication over shared artifacts on surfaces, such as a map on a table. However, in computer-supported distributed settings, deictic gestures can be difficult to see and understand. This problem can be solved through visualizing hands and arms above distributed surfaces, but current solutions are computationally and programmatically expensive, rely on a limited understanding of how gestures are executed and used, and remain largely unevaluated with regards to their effectiveness. This dissertation describes a solution to these problems in four parts: 1. Qualitative observational studies, both laboratory-based and in the wild, that lead to a greater understanding of how gestures are made over surfaces and what parts of a gesture are important to represent. In particular, these observations identified the height of a gesture as a characteristic not well-supported in distributed groupware. 2. A description of the design space available for representing gestures and candidate designs for showing the height of a gesture in distributed groupware. 3. Experimental evaluations of embodiments that include the representation of gesture height. 4. A toolkit for facilitating the capture and representation of gestures in distributed groupware. This work is the first to describe how deictic gestures are made over surfaces and how to visualize these gestures in distributed settings. The KinectArms Toolkit is the first toolkit to allow developers to add rich arm and hand representations to groupware without undue cost or development effort. This work is important because it provides researchers, designers, and developers with new tools for understanding and supporting communication in distributed settings.