Multi-Touch Computer Interaction and Task Management
Project Goals
Using a multi-touch interface as an accessory for a "conventional" workstation, create the necessary hardware and software to allow the user to manage minimized windows in free-form piles.
This goal can be broken down into two sub-goals:
- Create a computer display capable of sensing multiple finger presses (a multi-touch display), and integrate it with a conventional computer workstation setup.
- Create novel user interface software that improves the user experience of managing windows and tasks, considering the advantages and disadvantages of this interface device.
An Introduction to Interaction Technology and Multi-Touch
Multi-touch devices are recently among the most popular uncommon designs for human-computer interaction. The Microsoft Surface table and the Apple iPhone are recent commercial multi-touch devices, and the film Minority Report contains Hollywood's interpretation. Recent work in computer vision has made it possible for individuals to build the hardware required for optically-based multi-touch screens, leading to a boom in experimentation. Enthusiasm for hobbyist multi-touch was triggered by the 2005 paper and subsequent presentation by Jeff Han (NYU) on one such construction method.
Multi-touch displays provide the ability to directly, manually manipulate objects on screen, a useful benefit. However, there are still advantages to a keyboard (tactile orientation and feedback), a mouse (pixel-precise pointing), and a regular display (high resolution, improved ergonomics). Nevertheless, most investigators are using a multi-touch screen as the only interaction device, often as a "table computer." Furthermore, software using multi-touch input is often either limited in its use or more like a tech demo than an advance in usability. These are the factors that led to the specific goals of this research.
Final Project Deliverables
- Hardware:
- Standard computer set up with Linux and dual-head 3D video card for multi-touch use
- Webcam converted for infrared use in multi-touch desks
- Multi-touch computer desk #1, using projected rear DI
- Multi-touch computer desk #2, using LCD laser light plane
- Software:
- rp-mt-scripts - Software system/platform to prepare a basic Ubuntu Linux installation with all software required for multi-touch use, as well as maintain and run that software
- tactus-navigator - Multi-touch window navigator based on a "navigation display" concept - includes interaction design, base class implementation, and implementation of basic gestures.