Hardware Overview
Updated May 3, 2009 - Click here or scroll to bottom for links to older versions of this page.
Synopsis: To the user, the final product will look like a regular desktop computer, but with the addition of a multi-touch display on the same level (and horizontal) as the desk surface, located conveniently near the keyboard. The multi-touch surface will be built in to the desk. This display will be used only to run the software developed to permit users to easily manipulate applications and windows (tactus-navigator) and optionally other multi-touch applications.
Because projects involving hardware construction naturally run into more budget constraints than purely software projects, my hardware goals included making use of otherwise-unwanted surplus, outdated, or damaged materials that can still serve the desired function in this specific custom use, when possible. Nevertheless, ease of assembly and polish of the final product are important considerations in hardware costs. Reducing required assembly work, as well as possible openings for error, were considered when making decisions.
Principles of Optical Multi-Touch Sensing
All optical multi-touch sensing techniques are based on a few shared principles. Computer webcams are usually placed behind a touch surface and screen so that the whole screen can be seen. Lighting and a touch surface are selected so that when the user touches the surface, their finger either reflects more light or blocks incoming light. Computer vision software processes the webcam video, extracting "blobs" -- smooth shapes correlating to each touch.
To prevent the displayed image from interfering with the optical touch sensing, infrared (IR) light is used to detect touches. This works because IR is invisible to humans, and computer displays do not emit or block IR. Webcams are actually more sensitive to infrared than to visible light. To convert one into an infrared-only camera, its IR-blocking optical filter must be removed and replaced with a visible-blocking filter. Computer vision multi-touch devices built in this way can be affordable and easy for a hobbyist to build, since the only part connected to the computer, a minimally-modified USB or Firewire webcam, can be purchased cheaply and set up easily and without risk to the computer system.
For an overview of these device techniques, see the NUI Group Wiki - Hardware Page, or a more complete and comprehensive summary of touchscreen technologies can be found here (mirrored locally) - both of those are external links.
Blob Tracking and Calibration in tbeta
Download video or open in other media player
Desk and Computer Workstation
Two desks were available -- one taller and heavier one, and a smaller, foldable one, originally used as a sewing machine desk. The sewing machine desk will serve as a portable multi-touch device ideal for presentations and demos. Early hardware assembly and demos took place in this desk, but were moved to the larger, heavier desk ("Desk 1") for permanent installation.
A standard Linux-capable computer with the ability to drive the navigation display as well as the standard one was the target of my work. The computer used has these specifications:
- Pentium 4 2.4GHz processor
- 1 GB SDRAM (upgraded by SNC Tech Support Services)
- 20 GB hard drive dedicated to Ubuntu Linux 9.04 - additional part courtesy of computer science department (formerly used for Cofrin 112 Linux machines)
- BFG nVidia GeForce 8400GS 512MB dual-head graphics card, PCI - upgraded part purchased for project by computer science department (integrated video was single-display only and not compatible with required OpenGL features)
Click to play embedded video: screencast instructing how to set up Linux for dual screens using an nVidia graphics card. Download video or open in other media player
Hardware Device #1 - Projected Rear DI
This technique involves flooding the back of an acrylic sheet with diffuse IR light. The computer display is also projected on the rear of the sheet. As a finger approaches and touches the acrylic panel, the amount of IR light being reflected back to the camera beneath the surface increases. While this produces a relatively low-contrast image, image processing software can filter it to extract touch locations -- "blobs".
Implementation Status: Completed successfully - more information available
Hardware Device #2 - LCD LLP (Laser Light Plane)
The second device will construct an LLP (laser light plane) device using a deconstructed LCD, thin sheets of acrylic, and IR line lasers. The LLP technique provides a higher contrast touch image, and is generally simpler to assemble. However, laser safety concerns led me to meet with Dr. Michael Olson (Physics) to discuss feasibility. We devised a plan for safe implementation of the device, and he agreed to support this device development. However, due to startup delays, the laser parts did not arrive for this desk until shortly before the project presentation, so the construction of this extra desk will take place following "content freeze" of this web site.
Implementation Status: Partial, will complete - more information available