Project Journal
1/30/08
Since this is my first journal entry, I have some catching up to do...I received the project description from Dr. Pankratz a little over a week ago. Through a few meetings, we cleared up the bit of confusion I had and clarified the goals of the project. Since then I have been working on this web site (I always wanted a reason to teach myself CSS!) and getting the necessary hardware and software to work on the project. Looking at the last year's senior project of John Moss I began to investigate OpenCV as a library to help with the motion tracking portion of the project. Unfortunately, my laptop which will be used for the project is currently running Windows Vista which OpenCV does not appear to like. I also have a Linux partition (openSUSE and Ubuntu) but I cannot get the laptop's embedded webcam to work properly on these operating systems. I got a webcam from Dr. Pankratz and will be trying to use that with OpenCV on my desktop running Windows XP.
Also today Dr. Pankratz and I ordered an iRobot Create to use as the robotic part of the project. The Create is produced by the same company as the Roomba, and is basically the same thing with the vacuum capability removed. It is designed for projects such as this where people would like to control just the robotic aspect of a Roomba.
Update: I successfully installed the webcam and OpenCV on my desktop, however OpenCV requires a copy of Microsoft Visual Studio 6.0 or 2005 be installed on the same computer. My desktop currently has Visual Studio 2003 which will not work. After doing some research it appears I need to modify my system a bit according to these guidelines.
Next step: Remove Linux and install XP on laptop, learn more about OpenCV and its capabilities and begin writing motion tracking software, wait for Create to come in...
2/03/2008
This past week has been very busy for me so progress has been slightly delayed. I tried getting my Visual Studio 2003 to work with OpenCV, but the directions I found didn't seem to work. I looked online for more help and found some other resources I am going to try. Currently I am downloading updates for Windows and Visual Studio so everything I have is up to date.
I'm going to remove my Linux installation and will be putting Windows XP on the laptop. Once these two solutions are working I will be able to begin developing the motion-tracking software. The Create should be coming in sometime later this week and then I can begin working on the robotic part of the project.
Next step: Finish setting up OpenCV on systems and begin developing motion tracking aspect of project. If I cannot get this to work, I will revert to developing strictly on the computers in the Cofrin labs which have Visual Studio 2005 installed on them. Also, I may buy a copy of Visual Studio 2005 for my personal use so I can develop outside of the labs.
2/10/2008
Well, there is some good news and some bad news regarding my project. The iRobot Create that we had ordered as a large part of the project is on backorder and will not be arriving nearly as soon as expected. While this does throw a kink into the plans, it does not totally ruin things. At this point I have the other parts of the project that I can be working on (webcam motion detection, publishing video feed to the internet, and internet control of the robot). If the robot does not come in soon enough, I may just use a Roomba instead of the developer-friendly Create. Either way, I will have a robot that I can use for the project (although the Create would be really cool to have!).
This weekend I made the jump and formatted my laptop to have Windows XP on it. The install went fine, however I can't get drivers for all the hardware so my laptop is only about half functional right now. On the puls side, the embedded webcam appears to be working fine and I installed the copy of Visual Studio 2005 that I got from Dr. Pankratz on the laptop. I tested the webcam with John Moss's motion detection software and it captures the video properly, however of the two windows that popup (one for displaying motion, the other just the video feed) only the one that is selected gets updated whereas on my desktop both videos were updated together. I'm not quite sure what would be causing this, and it may need to be looked into in the future. I tried to install the external webcam that Dr. Pankratz gave me, but John's software still used the embedded camera for the video feeds.
Next Step: I need to create a project outline for the remainder of the semester. I already feel myself getting behind and I think a timeline will help me stay on track better (or at least have a better understanding of how far behind I am!). I also think I am going to try the setup again after re-installing Vista on the laptop. I'd really like to have a fully functional laptop for the rest of the semester and since HP doesn't support all it's hardware in XP anymore, I may need to revert back to Vista to get things working. I think I've figured out why openCV didn't work on Vista before so perhaps I will be able to get it working. Either way, I need to make measurable progress on the project this week or risk falling way to far behind.
2/17/2008
This past week I started working on the coding of my project. I have created a simple Win32 program that displays a web cam feed using openCV function calls I have also done a lot of research into motion tracking algorithms using openCV. In meeting with Dr. Pankratz we decided that the robot should automatically move to keep the focus object in the center of the camera's window rather than wait for it to move nearly out of the window before adjusting. Also we discussed some other design decisions that need to be addressed when that part of the project is reached. I talked to Dr. Blahnik more about using the webcam in a VB program. He has a dll made in C++ that calls some basic OpenCV functions to stream a webcam image to the VB program. I may use something like this when creating my server program.
I also began looking into ways to control the Roomba through program control. Although I have not been able to completely get this to work yet (mainly issues dealing with the proper ports and protocols), I do have a good lead into this part of the project.
Next Step: I need to continue with the progress I made this week. I would like to get a working motion-tracking program done this week and have solid direction on controling the Roomba under .Net or 6.0 program control.
2/24/2008
Another case of good news/bad news about the project this week. The bad news is that I was busy in some other classes and things and didn't get to work on the project as much as I would like to. The good news is that the things that I did accomplish are all important steps in the development of the project. First off, I was FINALLY able to get my laptop back and running with Vista installed on it. I may not be able to develop much using it yet (since I'm still using Visual Studio 6.0) but having a laptop to execute the program is a crucial part of the process so it's good to know that I have a functioning laptop again. Also, I was able to convert the camshift demo that comes with OpenCV from just a .c file to one that is incorporated into a development solution. This means that I can now use that program as a basis and modify it as I need to. That being said, after doing some testing and research on the camshift algorithm, it may not be the right algorithm to solve my problem. The camshift algorithm relies on the color of pixels to determine movement. While this is a valid and useful way of detecting motion, it may not be suitable for tracking motion around an environment (a home for example) that has different lighting conditions. The change in lighting from one room to another or even from direct light to darkness may greatly alter the results of the motion tracking algorithm. I'm looking into other algorithms that are incorporated in OpenCV that track shapes or lines rather than colors. This would probably be a better way to track the motion while moving around the environment.
Next Step: Find a better motion tracking algorithm, determine whether or not creating C++ dll's to call from a VB 6.0 program is the route I want to go down. Fix the situation with the port ambiguity and create a dummy program that controls the roomba.
2/25/2008
Just a quick note - I've added a diagram of how the final project will look like. It's located in on the "Project" page.
3/05/2008
Yes another good news/bad news update. The good news is that the iRobot Create was shipped and will be here very soon. Hopefully this will be what I use as the robot part of the project. It's intended for people to develop on and has some added functionality that the regular Roomba doesn't. The less good (bad) news is that there are more difficulties with the motion tracking algorithms I'm working with. In talking with Dr. Pankratz, we're not sure how to continue on with the motion tracking aspect of the project or if to continue on it at all.
3/07/2008
My project has been modified a bit based on the difficulties in motion tracking. Creating a successful motion tracking algorithm for a moving camera would be a project in itself, so my project is being changed to remove the motion detection/following part. Now, the project will entail controlling the robot through a program on the server laptop that it is connected to. This computer will then publish the camera feed to a client computer via a wireless network. This client computer will also have an interface so it can control the robot remotely across the network. I will be updating the Project page and the rest of this site to reflect the change in my project.
Also, my walkthrough for the project is coming up this Tuesday. With this short deadline I need to get a full understanding of the new project and begin to develop my solution so I can present it on Tuesday. I'm hoping that though the presentation and discussion I can get some good feedback and ideas for the future development and direction of the project.
Next Step: Now that the Create has come in, I need to start working on that to get better understanding of the control of the robot. I'd like to have a working model going for my presentation on Tuesday so this is my first priority as of right now. I also need to update the rest of this site to reflect the changes that have been made in my project requirements.
3/09/2008
I have updated the web page to reflect the changes in my project definition. As of right now I am keeping the Web Interface page in, but I am not sure that I will be needing it anymore. I spent some more time working with the Create robot and understanding the navigational commands. Also, I have been thinking about the new project and have a basic design of how I will shape the system.
Next Step: Prepare for walkthrough on Tuesday!
4/08/2008
Well, it's been a while since I've updated things here. Lately I've been making a lot of progress on the project and most of the major hurdles seem to be behind me. Currently I have a fully functioning VB program that controlls the roomba through the serial port. This will become the client (remote) program in the final solution. Also, through working with Dr. Pankratz, I now have a socket program that runs between two VB programs on separate (or the same) computers. This will be used to communicate between the two computers in the final setup. As of right now I am begining to develop the server application and modify the client application to work with the sockets. The client will send a code across the socket when the robot needs to be controlled. This will be accepted by the server which will uncode the message and send the proper commands to the Create. I have a basic version of this program working currently and it seems to be working perfectly. Also (with my dad's help) there is now a base to hold the laptop on top of the Create. It allows the laptop to be elevated and not affect the bump sensors and also prevents the laptop from tipping.
Next Step(s): Finish developing the code-language used to communicate between the two programs. Finish development of server program to handle client commands. Work to set up a wireless network and be able to establish the socket across that network.
Update: I also updated the Resources page with some of the resources I've been using so far. Some of the internet resources are not in use in my solution any longer since I switched away from motnion tracking, but they are still great resources for OpenCV development.
4/12/2008
Today I was able to make a lot of progres on my project. I finished the development of the client and server programs that communicate between each other to control the Create remotely. These programs use the psudo-language that I developed to communicate between the two programs. This language is basically a system of coded integers that represent different movements of the Create. The client/server interaction now seems to be working fine using a wired connection for the laptop.
Next Step:Work to transmit the video feed across another socket, work on the communication between the programs through a wireless router.
4/21/2008
I've been working a lot on the project lately and haven't really updated this page with all the info. I think I'm going to wait until I have a final version before I list all the updates mainly because right now I need to focus on making as much progress as possible and finishing up the prep for my presentation tomorrow night!
Next Step: Presentation tomorrow - 5:30 Cofrin 11
4/26/2008
Well, the project is pretty much over. I had my presentation last week and everything went really well including the live demo. I was able to get most of the requirements of the project functioning completely. I was even able to add in another form that gives the client/user the ability to control the robot via the keyboard arrow keys. This seems like a more intuitive way to control the robot. Although this interface does not have all the functionailty and options of the original mouse-based interface, it does work better for people who are unfamilar with the interface. This will be my final update to the site as the final copy is due on Monday morning. I have removed the Web Interface page that I didn't use and replaced it with a Project Demo page. On this page I published my powerpoint presentation along pictures and videos that demonstrate how the applications operate.
5/2/2008
Ok, so I lied...THIS will be my final update of the web site. I had my project defense this past week and it went really well. Overall I'd say I'm quite satisfied with the project. Grant it I'd like to add more to it and fix some things up, but that's only natural when you've been working on something so long. I updated the site some more, changing the "Resources" page to "Resources and Downloads". On this page you can download everything needed to run your own version of my project and even modify it for your own needs. I am also working on finailzing everything for the project. I need to print off all my code and examples, etc and present it formally in a binder and on a CD. I'll be doing that all this weekend and that should be the end of my capstone experience. I hope you've enjoyed the ride as much as I did and hopefully my work will be of help and/or interest to you!