Software Overview

Old Version! - Updated February 11, 2009 - Click here for the latest version

Original software: "Navigation Display" window manipulator

I will be constructing, in a mix of C++ and Python, a software system that replaces a dock or taskbar with a full-screen interface on a secondary multi-touch "navigation display". Below and to the right, you can see a mockup of the interface that will be visible on the navigation display.

Users will be able to move icons or thumbnails representing minimized windows on the navigation display freely, using direct touch manipulation. The bar at the top of the navigation represents the main display, and entire free-form groups ("piles") of windows can be "pulled down" or "pushed up" to the display by interacting with the multi-touch surface. The user will be able to use unmodified, standard Linux graphical software with this system.

Existing software used as a base

I will be using, when available, open source libraries to perform the basics of multi-touch processing and device handling, to permit me to focus on the higher level interaction. Furthermore, I will make use of open source software that manages windows as an example to learn from others' experience in the basics of window management and speed development.

  • TouchLib - "Blob-tracking" library, using the OpenCV open source computer vision libraries by Intel to analyze the infrared webcam feed from the device.
  • Sparsh UI - a flexible, cross-platform, open source multi-touch gesture recognition framework that can connect to TouchLib as an input backend, and provides recognition capabilities for C++ and Java programs.
  • Gnome Desktop Environment, as included in Ubuntu Linux - a powerful, open source, general purpose computing environment and operating system (comparable to closed-source Windows and Mac OS X), to be used as the implementation environment due to the ease of replacing the desired desktop component (window switcher) and the available open-source libraries.
  • Avant Window Navigator - an open source "dock-like" window switcher, to be used for window manipulation and icon access code when building my software