Threading fixed and presentation

4 May 2011

Here are the files from my presentation on the 28th of April:

• CMUcam3 firmware (112KB);
• CMUcam3 virtual cam executable for Windows, which was not used for presentation (88KB, requires Cygwin);
• Windows frontend (53KB); and
• presentation slides (1MB, PDF).

I was able to solve, for the most part, the threading issues.

Threading unsafe: C♯

18 April 2011

Now it is threading in C♯ that is giving me problems, but only sometimes. It seems that Windows Forms are not thread-safe. This would be perfectly reasonable, I am not explicitly using threads, except that C♯ seems to be doing this all on its own. When it does happen—the exception dialog warning of a cross-thread operation makes it quite clear—I am able to revert the code and it works as I would expect. From search on the Web, it seems that the SerialPort class uses threads.

A desktop companion

17 April 2011

Previously, I had been using PuTTY to monitor the serial port:

Since black windows are no longer fashionable, I decided to create a Windows frontend. I had initally planned on modifying the CMUcam3 Frame Grabber program but decided to start from scratch in C♯ when I saw that it is written in Visual Basic. Not having ready access to the source code of my previous projects in C♯ to aid my memory, I did not remember as much of the language as I thought I would have.

Euryale Desktop, what I am calling this endeavour, does little more than put the data sent by the CMUcam3 into labels:

My first thought for improvement is two-way communication. I have a few ideas on what this could be used for:

• finding a new colour to track;
• choosing a new colour to track; and
• enabling or disabling the servos.

Right now, the Euryale firmware sends new position data without waiting for an acknowledgement. I had looked into using threads on the CMUcam3, but newlib does not implement threading. The CMUcam2 emulation firmware is essentially an infinite loop that waits on a getchar().

Slides from walkthrough

14 April 2011

Here are my slides (PDF, 173 kB) from my walkthrough from the 24th of March.

Source code changes and the end of the spoon

13 April 2011

The eponymous spoon is finical. After having to glue it on several times, calibrating it each time, the last time I simply turned the micro-servo with the power off. Oopsie. As a consequence, I modified the spoonBot source code so that it can be compiled without having to use the micro-servo for the spoon.

I also modified the messages printed over the serial port to make them more consistent.

Hot glue gun fun

29 March 2011

I have started gluing the spoonBot together. The battery pack and the wheel servo-motors are glued together as are the micro-servo horn and the end of the broken spoon. I may have not left enough of the spoon handle; the bowl of the spoon might get in the way of the micro-servo motor. Prof. Pankratz, Prof. McVey and I had trouble understanding the directions for how the spoon is attached but I think we figured out a way to make it work. From the picture of all the parts, it looked like the servo horn had only one wing on it, but then in another picture it suddenly has no wings.

Direction forward

6 March 2011

Prof. Pankratz suggested I look at OpenCV, «a library of programming functions for real time computer vision», but after reading this post, and a few others, on the CMUcam3 forums, I am not hopeful:

OpenCV does quite complex image processing that would be difficult if not impossible to run on the CMUcam given the arm processor. OpenCV was designed to run on desktop machines. It is even difficult getting it to run on most DSP camera systems which have much more powerful processors than the CMUcam3. There is a trade-off between cost, size, power consumption, and ease-of-use that comes into play with more complex image processing.

My next step will be to actually construct spoonBot. Prof. Pankratz bought a hot-glue gun and a regulated power supply.

Face detection is no longer a requirement—the processing time per frame is too long.

Information collection

1 March 2011

I have begun organizing information from weblog and organizing it to the overview page.

A meeting with Pankratz

17 February 2011

I met with Prof. Pankratz on the fifteenth. As the CMUcam3 is unable to provide a live digital stream, we have to rethink the project a bit.

I have two pairs of chopsticks, but they seem too nice to cut up.

One bot a tracking

13 February 2011

I was able to compile and flash the Viola-Jones firmware onto the CMUcam3. It takes a picture and attempts to find a face when the pushbutton is pressed. If it finds a face, it prints its position and size to the serial port. It can optionally save the image to a memory card, but I do not have one to use.

Sourcery G++ woes and insufficient power

12 February 2011

I have tried compiling the CMUcam2 emulator with several different versions of the cross-compiler. 2006q3, which came on the CD–ROM from Seattle Robotics, has a runtime error. 2008q1-1-26, which is the latest version on the CMUcam3 website, hangs. 2010.09-56, which is the latest version on the CodeSourcery website, gives the error mentioned in my previous post.

It is very frustrating to have three different versions of the same software behave in all different, and all incorrect, ways when attempting to do the same thing.

I was able to compile and flash the spoonBot firmware on the CMUcam3. Internal power is insufficient for both continuous rotation servos, causing it to reset. It works as expected when only one of the two is connected.

Three servos serving

9 February 2011

Prof. Pankratz was able to find a jumper. Using the CMUcam3 Frame Grabber, I was able to make the servos move.

Motionless servos

8 February 2011

I received the servos today; I am unable, unfortunately, to test them. The CMUcam3 must either have a jumper set to move the servos from internal power or it must be connected to second power source. I will have to get one or the other from Prof. Pankratz.

I plan on using the CMUcam2 emulator firmware to test the servos before I start working on a custom firmware.

From looking at the directions to construct a spoonBot, I might have to find something to use instead of the battery pack—since I am using an AC adapter—for stability.

CMUcam3 arrived

7 February 2011

The CMUcam3 has arrived; I am excited to get started creating the robot. Prof. Pankratz gave me a USB to serial adapter.

The CMUcam3 development on Windows requires a few programs:

• Cygwin;
• the cross compiler;
• the memory flasher.

The Serial–USB adapter also requires that a driver be installed.

So far, I loaded the CMUcam2 firmware emulator onto the CMUcam3 and was able to use the CMUcam3 Frame Grabber application to grab images from the camera:

Image frames are not able to be sent via the serial port with anywhere near the frame-rate required for video.The analog video outputs are only pads and would need to have a connector soldered on.

I am having trouble getting the cc3 software, which includes sample applications for the CMUcam3, to compile. The Makefile had been hanging on CC cmucam2_lpc2106 but now, thankfully, it is at least giving me an error message from ld(1), undefined reference to __ctype_ptr, so now I will have something to go on.

Hardware purchased

1 February 2011

After consulting with Prof. Pankratz, we purchased hardware:

• from Seattle Robotics, a CMUcam3;
• from Acroname Robotics, two continuous motion servos, one micro-servo and a pair of wheels.

I will continue researching the development environment for the CMUcam3. Prof. McVey is able to disable Deep Freeze so I will not have to re-install the software after the lab machine restarts.

The CMUcam family

30 January 2011

The CMUcam3 is the successor to the CMUcam2 and the CMUcam:

Device	Cost	Comments
CMUcam3	$249	While the CMUcam3 can run the CMUcam2 firmware, it is also programmable.
CMUcam2	$180	A more powerful CMUcam with greater functionality. Has five servo outputs.
CMUcam	$110	Cheap and small.

I had assumed that all three would be programmable, but apparently only the CMUcam3 is; the others are black boxes. I understand the CMUcam2 and the CMUcam to require an external micro-controller while the CMUcam3 does not.

The CMUcam2 supports face tracking, which is the primary requirement of this project.

The CMUcam3 does not come with a firmware, though it supports running the CMUcam2 firmware. It has serial output, but I do not know if it can send a video feed. It also supports analog grey-scale video output—either PAL or NTSC, depending on the camera.

There are two precedents for this project, the spoonCam and viola-jones. The code for both are in the CMUcam3 SVN repository and are licensed under version two of the Apache License.

The spoonBot is a «simple table-top robot» built with a CMUcam3 that can pivot and pan.

The «CMUcam3 Face Detector» (PDF hyperlink), is based on the algorithm of P. Viola and M. Jones,.

The Beginning

27 January 2011

The project has been assigned today.

Given the title of «Motion Tracking», I am to «develop a bot that tracks a person’s movements and visually describes the activity». I have been given very little as to the what the project involves:

General Description and Requirements:

1. The bot is sensitive to at least two dimensions.
2. Detection of depth (distance).
3. The bot might look up and down as well as left to right.
4. Warn about boundary conditions like out-of-bounds and too many people.
5. Perhaps it keeps the person centered on a screen.
6. Develop a way to activate the bot for a user and to let it sleep when lonely.
7. Consider a playback option.
8. Perhaps speech output as well as visual.
9. The bot’s home should be the display cabinet in Cofrin to help showcase computer science.
10. Look into the CMUcam 3.

Most significantly, the hardware for this project is undefined.