First off, I must say that I am very excited about my project assignment. Wireless technologies and GPS have always been very interesting to me, and I am looking forward to building a system that combines the two to create a sort of real-time tracking device. Additionally, I think it is neat that the results of my research and work this semester will be used by future students to build some sort of robot or remotely-operated vehicle. I think my designing and building a robust tracking and communications system will lead to really neat additions and projects in the future.
Since the project assignment on Thursday, I have been working on a high-level design of the project. Additionally, I have been assembling a ‘laundry list’ of what hardware might be required.
The first high-level design that I came up with uses a cellular phone to connect the mobile device to the Internet and then uses ports (or sockets) to communicate to the server. Here is an overview:
Laptop runs a program that dials up connection to Internet, connects to a socket or port on COMPSCI, and then continuously (every 1 second) reads position from GPS unit. Whenever it gets a reading, it sends that via some protocol to COMPSCI. COMPSCI then has an application running on it that continuously monitors that port and stores the latitude/longitude into a database table whenever it is received. To accomplish two-way transmission, there could be a ‘receiver’ program running on the laptop that receives commands and a ‘sender’ process on the server that pulls the commands out of a queue and sends them. The web page (a perl program?), then would have two purposes: to read latitude/longitude/timestamp entries out of the database and display them (map them maybe?) and also to send commands to the laptop.
The troubles I am seeing with this system primarily involve the ongoing cost of using the device. This design requires a dial-up account with ITOL or some other Internet Service Provider (which carries a monthly charge with it), a monthly cellular phone rate plan, and a per-minute charge on the cellular phone to connect to the Internet. Although we may be able to receive some educational discounts on these services, I would like to consider other designs or means of communication that might be less costly.
One alternative to the design presented above would be to have the mobile device dial directly into a server at SNC that accepts incoming dial-up connections. The two devices (client and server) could then communicate via a standard modem protocol. This would eliminate ITOL from the picture. However, it would limit the number of mobile devices to the number of modems and phone lines connected to the server. Although we will begin with only one device, the future may call for multiple devices to be tracked at once. Another drawback to this method is that it does not use standard Internet ports (or sockets) and it would therefore be difficult to migrate it to a different type of connection (such as dial-up to Internet) at a later date.
My third design, which interests me the most and eliminates the ongoing connection costs, is to use Wireless Ethernet to connect the mobile device and the server. This technology has recently become affordable for businesses and consumers and is being used across the country to share Internet connections and expand networks beyond the reach of wires. The technology is also referred to as IEEE 802.11b and Wi-Fi. To set up a wireless Ethernet network, you must have an access point (similar to a router or hub on a wired network) and wireless network cards for every PC or laptop on the wireless network. The access point can then be plugged into a conventional wired network (the SNC network).
In our case, we would have a wireless access point plugged into the campus network at a strategic location. The laptop would then have a wireless Ethernet card installed in it, and it would then be able to access the campus network (and the Internet) whenever it is close enough to that access point. The only drawback to this design is the limited range of the access point – based on web sites that I have looked at, the maximum outdoor range is 1000-foot radius per access point and the maximum indoor is 150-300 foot radius indoors, depending on building construction.
I still need to look into the cost of access points, cost of wireless network cards, and the number of access points required to cover the campus. I also need to talk to DCP about the final expected range of this robot/vehicle/system – if it is the entire Green Bay area, we cannot use it. However, if it were expected to remain on-campus it would be adequate. This sparks another idea – what if we implemented both systems, but only engaged the cellular connection when the access point was not available? Clearly I need to run through this all with DCP and see what he thinks about the range of this thing...
Costs eliminated by using wireless access points:
I looked at the current inventory of used laptops at http://www.aplaptops.com. Currently, they have a number of laptops that would serve our purpose in the $300-$400 range. One that really caught my eye was the Itronix Ruggedized laptops they have in stock right now. These are intended for heavy-duty use, such as on construction sites. They have reinforced hinges, extra-shock-protected hard drives, and a durable magnesium alloy case. I sent the company an inquiry about the presence of USB ports on these laptops and am awaiting a response. The price listed on their website for these laptops is $399. I should have more information next week.
Other Random Thoughts:
We will likely need two COM ports on the laptop – one for GPS, one for Phone / Modem. Laptops rarely have two COM ports, so we will likely have to get a laptop with USB and buy a USB->Serial adapter. I priced out PCMCIA->Serial adapters on the net and they are WAY too expensive - $300 and up! USB->Serial adapters can usually be found for ~$30.
Should the daemon that is running on the COMPSCI box listening to the port do the actual inserting into the database or should it fork a thread to do that every time? We don’t want it to get slowed down and miss data or lose data as it is arriving.
The web page shouldn’t be too difficult, I don’t think. For now, we can read the most recent timestamp from the database and display it on the screen. The page should have a meta refresh that reloads every second and gets a new version. I will need to find out how to access an OBDC / Oracle database using PERL.
Mapquest can map lat/long with this:
How am I going to control Dial-Up Networking with C/C++?
How can I do sockets using Visual C++?
Today I followed up on the Ruggedized Laptop idea with a phone call and found that it has one Serial port and no USB ports. I think it is a good idea to get a machine with USB ports so we can use USB->Serial adapters to hook up as many external devices as we can. Right now, we are considering the use of a GPS and a Cellular Phone, and in the future we may want to control external devices with this one laptop.
Things we (DCP?) need to figure out:
Questions for DCP:
Issues raised at class presentation / overview
Tonight I priced out multiple levels of GPS units and Laptops for tomorrow’s meeting with Dr. Pankratz. The specs on the GPS Units I priced out can be found here and the specs for the laptops can be found here. I also found out about Packet Radio Communications, which allows you to transmit data using Ham radio airwaves. This, like the wireless Ethernet method of connecting, would not cost anything after the initial hardware was purchased and would likely have a greater range than the wireless Ethernet, especially if we were to use a repeater that is hosted by a local amateur radio club. I have emailed the Green Bay area amateur radio club that meets on campus (never heard of them, surprisingly) as well as Jim Lienau who was on the Door County Amateur Radio Club membership roles and asked where I might be able to get quick training and administration of the exams required to get my license. From what I have read on the Internet, certified HAM operators can certify new operators and the license required for this method of communications does not require me to learn and morse code. I will keep reading up on this and see if I can find someone to help me out with this if I am able to get my license in the next couple of weeks.
Tonight I also taught myself some Cascading Style Sheets for use in developing my web site. These allow you to really lay things out easily on your page and keep a common look and feel throughout your pages. Luke was working with them in class and I liked the look of what he was doing so I sat down for a few hours and now feel pretty good about them – I will continue to learn as I go, I’m sure, but I made a very professional-looking front index page that I can now model the ‘content’ pages after.
Today at a meeting with Dr. Pankratz and Professor Blahnik, we decided to go with the Garmin GPS76 and to get the combination data/power cord with it to eliminate the need to deal with AA batteries when experimenting. We also chose to purchase an IBM Thinkpad 390X from aplaptops.com. I inquired about purchase orders at this site and they only will accept checks, money orders, or credit cards, so I will talk to DCP about ordering that with a credit card in the next couple of days.
Regarding the GPS unit, I called Wal-Mart and they do not carry it in their store here in Green Bay, although it is offered on their web site. I also called Gander Mountain, who normally carries it but does not currently have it in stock and does not know when they will get it. I also called Best Buy, who does not carry that model. Finally, I called Fleet Farm, and they do not carry that unit either. I think our best bet would be to purchase the unit from Wal-Mart.com or westmarine.com.
In our meeting, we also discussed the data transfer method and I told them about my two ideas, Wireless Ethernet and Amateur Radio. We concluded that the easiest way to do this right now would be with the cellular phone dialing in to the Internet via ITOL, so that is what we will pursue. Perhaps someone can convert the system to a more cost-efficient design at a later date. This will allow us to get up and running as soon as possible.
After the meeting, I went to talk to Ms. Gibson about ports on COMPSCI. We reserved port 3886 and she sent an e-mail to Rob Downard requesting that port be opened up at the firewall so we can communicate to COMPSCI from outside the network.
Today Dr. Pankratz and I ordered the laptop, a 12V DC Car Cord for the laptop, the GPS Unit, a Power/Data cable for the GPS, and a carrying case for the GPS. When those items arrive, I will be able to start reading data off of the GPS unit. I also will soon begin learning how to implement sockets to access ports in Windows and Solaris, and I will also look into establishing a dial-up Internet connection via C/C++ in Windows.
Today I also received word back from some HAM/Amateur Radio people in the area regarding the licensing required for HAM radio which can then be used to send packet data between two stations. I had originally thought that this might be a possible means of communications from the mobile unit to the server, but we decided it would be a lot easier to use the cellular phone / dial-up Internet connection instead.
It seems my project has reached somewhat of a lull - I have most of the design and planning down but I am waiting for the GPS Unit and Laptop to arrive so I can really start working on the protocol for getting data off of the GPS and also on establishing a dial-up connection through C/C++. Hopefully those items will arrive this week!
Update - 11:15PM: Tonight I found and downloaded the PDF that explains the entire protocol for communicating with the Garmin GPS Unit via the serial port. Although Professor Blahnik has succesfully communicated with a Garmin GPS Unit before, I believe he has done it in Visual Basic, and I think I will be doing it in C++. I am thinking I will write a driver layer for the GPS Communication so my higher-level program can simply request the current coordinates without dealing with the nitty-gritty protocol issues, handshaking, etc.
I have an appointment for tomorrow morning to talk to Ms. Gibson about installing Perl modules on COMPSCI so I can communicate with an Oracle database from my Perl program that will generate the dynamic web page that will display the current coordinates of the Mobile Unit. I also intend to ask her about "cleansing input" from the page that refers to the CGI program - I think I have an idea of what this is but I'm not entirely sure so I will go through it with her. I think it has to do with security and making sure no one uses my Perl program to get access they shouldn't to COMPSCI.
After that, I will meet with DCP to talk about ports/sockets on Solaris and what he knows about them. I will also discuss with him my plan for storing coordinates in a database as they arrive and ask him what he thinks about that. It sounds like a good idea, but perhaps it would be faster (less overhead) and easier to simply write the coordinates to a local file instead. Speed may be essential in this situation because the coordinates will be continually arriving at the server from the Mobile Unit and we want the server to be able to store data quickly and be prepared for the next set.
This morning I met with Ms. Gibson and found out how to cleanse input to my Perl scripts to keep anyone from using them to access the system illegally. She also informed me that she would install any required Perl modules in the global area of the system if I tell her which ones I need.
I also met with DCP, who had all of the things we ordered last week waiting for me! We talked about career-related things and also discussed how data will be stored by the listener on the server and then retrieved by the web server for display to the web clients. I need to do some experimentation to see what kind of blocking the OS will do to a file that is accessed for writing by a C program and reading by a Perl program. I will create a file, open it in a C program, and then try to open it for reading with a Perl program - hopefully the OS will block the Perl program until the C program closes the file.
Finally, DCP gave me some books about Sockets (both in Windows and on Unix) so I can begin working with those. I will be able to use one of the Linux machines in the lab and COMPSCI to do some testing on this, and in addition I can use the ITOL server for experimentation off-campus.
It will be fun to get my hands on this equipment and start seeing what we can do!
Update - 7:00PM: Tonight I started 'playing' with my new 'toys'. I installed Microsoft Word and Visual Studio on the laptop and I started using the GPS a little. I downloaded the latest firmware for the GPS unit from the GARMIN web site and installed that, so I know the serial interface is working. I haven't yet tried the GPS unit outside, so I haven't been able to lock onto enough satellites to get a good location, but I found that the unit has a simulator mode where it estimates your position based on your last known position and operates just like it would if it did have a signal. This is very helpful for me because it means I can develop inside! I can put it in simulator mode and read coordinates just like I would in 'regular' mode. Tonight I also printed out the entire GARMIN protocol specification document off of their web site, so hopefully in the next couple of days I can begin talking to the unit myself! That will be very exciting and I am looking forward to it!
This past Friday, I went to Professor Blahnik to see what he has done to communicate with his GPS unit, which is also a Garmin but is a different model than the one I am using. He told me that he had done everything in Visual Basic, and although I was thinking about writing the program for the Mobile Unit using C/C++, I am now reconsidering the use of VB. VB would allow me to get an app running quickly and easily, without too much worrying about minor details when writing to/from COM ports and sockets and when initiating and managing a dial-up Internet connection. I will run this by DCP early this week before starting to make sure this fits in with the long-term design and usage plans for my project.
Other than that, I didn't get much time to work on the project this weekend. I am meeting with Professor Blahnik on Tuesday to get his VB-Garmin source code from him and to discuss the protocol and what he knows about it.
Last night (this morning!?) I broke out my notebooks and texts from Operating Systems and Programming Languages and wrote two small experimental applications - one in C that opens a file for writing and then sleeps for 10 minutes before closing it, and another in PERL that opens the same file for reading. This is to determine how much blocking the OS handles on files and how much we will need to handle (because we will have a C program monitoring the socket and writing to the file whenever it receives data and also a Perl program that will be run any time a web client accesses the web page that reads from that data file). Although it's pretty late and I may have missed something, and I can think of some better test/experiments to run than this, it appears that the OS will allow the Perl program to open the file for reading while the C program has it open for writing. I think now I should make the C program continually write something to the file (not just have it open) and have the Perl program continually read to make sure it is getting the up-to-date data, etc.
I'm tired and going to sleep now - more tomorrow, hopefully!
UPDATE: 10:00PM - Tonight I modified those file reading/writing programs to better represent how my system will eventually work. The C program was modified to open the file and write the timestamp to the file every 1 second (much like it will write the incoming data to the file) and the Perl program was turned into a CGI web page that will refresh every 1 second and display the contents of that file to the web client. However, when I went to make the Perl program available to the browser on the Internet, I kept getting an Error 403 Forbidden page. I contacted Ms. Gibson about this and will hopefully have that working properly tomorrow. So far, though, it looks like all blocking and read/write coordination will be handled by the OS, which is kind of nice.
Tonight I also set up the dial-up connection to ITOL on the laptop (using my family's login) and the internal landline modem. After that, I found some VB code examples of how to establish and disconnect a dial-up Internet connection. This is done with one function call from a built-in library - pretty easy and convenient! I obtained that information from http://www.thescarms.com/VBasic/ShellExecute.asp.
Tomorrow I meet with Mr. Blahnik to talk Garmin and the GPS Protocol - how exciting!
Today I met with Mr. Blahnik and we discussed the Garmin protocol. He showed me his VB program that he uses to talk to his GPS Unit (which is a different model than mine) and provided me with the source code for this to use as a basis for my own program. We then plugged my unit in and tried it with his program. The current location was obtained properly, but my unit did not respond well to requests for the current time, stored waypoint information, and track information. So, we started looking at the packets that were coming back after the request for the waypoint data was sent, and found that they looked similar to those sent back by his unit yet they had some very large differences. Looking at the packet that was getting sent back to the PC from the GPS unit byte-by-byte, we were able to determine where in this packet the name of the waypoint was stored. We then compared this location (56 bytes into the packet) with the packet structures listed in the Garmin Protocol specs but could not find a packet with variable-length (null-terminated) names or a name field 56 bytes into the structure. Since the last published specs are from December of 1999, we concluded that my unit uses a waypoint protocol/structure that is newer and different than any others listed in the specs. So, we will now need to look at a number of waypoint packets and try to figure out what data is stored where in them.
Although Mr. Blahnik would like to eventually obtain waypoint data from this unit, my project really only requires the current latitude and longitude (and maybe altitude). So, I think I will concentrate on that (or the Position Velocity Time (PVT) data) for now and worry about waypoints later.
Today I also worked with my experiment on file blocking a little more. I modified the Perl program to open the file for reading and then sleep for a long time. I then started that program and then started the C program. I discovered that the OS allowed the C program to write to the file even though the Perl program had it open for reading. I'm not sure what to conclude about that - after yesterday's experimentation I'm beginning to think that our reading and writing once per second is so relatively long compared to CPU cycles that we may not need to worry about it. However, it is good to be thorough and sure that everything will work properly. I will talk to Dr. Pankratz tomorrow about my results and see if he agrees.
UPDATE: Tonight I worked on the Garmin Protocol a little more and discovered the packet structure for the waypoints that Mr. Blahnik and I couldn't figure out today - so even though my project won't require reading waypoints, we will be able to eventually add that capability. I also decided to try to write the communications layer in C++ because I am so used to manipulating byte-arrays in C/C++ and I found a COM port class from work that I can use to communicate easily. I may eventually convert this C++ into a DLL and call it from VB, but we'll see. In C++, I was able to send a request for location and get an ACK packet for that request, but I don't seem to be getting the data packet that follows the ACK. Maybe I can talk to Mr. Blahnik about this tomorrow.
Tonight I also installed the new Ethernet Adapter in the laptop and registered the MAC address so it can now connect to the Internet and the Campus Network.
ANOTHER UPDATE: I figured out what was wrong with my Garmin Protocol C++ program, it was a protocol 'issue', and I now am able to request the current location from the GPS unit and display it in degrees on the screen. I think that's quite a milestone! Of course, I need to clean this up into one GetPosition function and I would like to look into this PVT mode that dumps position, velocity, and time constantly, but for now I can move on to working with sockets and establishing communication between the laptop and the server.
Today I met with Dr. Pankratz and showed him the sample program that reads the current coordinates off of the GPS unit. We discussed the C / VB issue, and decided that I should write the driver layer in C/C++ and then create a DLL which can be called from a VB program with a nice user interface. C/C++ just seems like a naturally better choice of languages for this application because the spec sheets from GARMIN use C/C++ notation to define the structs and other data types. We also discussed getting a mouse for the laptop I am using as well as a hub so I can plug it into the Ethernet in my room along with my personal desktop PC. Finally, we decided that the next thing I should tackle is the use of ports and sockets. So, I will begin working with those later this week or next week and will then take what I have to him if I am running into any problems.
I haven't done a whole lot with the project since my last journal entry because I have been very busy with other classes and projects, but I did get a mouse and hub from Dr. Pankratz. My goals for this week are to continue working with the GARMIN protocol to develop my driver layer and to begin experimenting with sockets. I also need to sit down and do some design of the driver layer, which will include determining what options I should give to the program that is using my drivers. Off-hand, I can think of get current position (raw data, radians, degrees), initialize unit, and maybe get PVT (position, velocity, time) but I need to find out exactly what that is before I'm sure of that.
Today I finally got my socket programs (client and server) to compile on COMPSCI, and they work! All I am doing right now is sending characters from the client to the server, which capitalizes them and sends them back to the client, but that's better than nothign! Now I need to start looking into sockets in Windows for the mobile unit. Hopefully it won't be too difficult getting the Windows sockets to talk to the Unix sockets, but I guess I won't really know until I try!
Today I started looking at sockets in windows and found them very surprisingly similar to sockets in UNIX. I thought that the concepts behind them would be the same, but the syntax, naming conventions, and data structures are also nearly exactly the same. Although I haven't yet established communications between the Windows Client and the Unix Server, I can't imagine I am very far away from that. Maybe this weekend...
Well, tonight I got a Windows/Visual C++ client to connect to my Unix server using sockets! Now if I combine the GPS data reading program with the socket client, I should be able to transmit the location data back to the server! In theory, I'm only a short while away from a VERY beta version of my system. I think I am also getting close to the end of my list of 'unknowns' or things I didn't know how to do before. Once I get this beta version working, I plan to work on system design and then start implementing a solid, expandable, well-documented system.
Tonight I created a working beta tracking system! This system consists of three components: the client program, which runs on a PC that has the GPS unit attached and, every 3 seconds, reads the current location and sends that location via sockets to the server program. This server program runs on COMPSCI, a Unix machine, and listens on a port for a location to arrive. When this location arrives, it formats it in degrees (from raw data) and writes the location, along with a timestamp, to a data file on the server. The third and final component is a Perl script that can be accessed as a web page via a web browser. This web page reloads every 1 second, and every time it reloads, it goes out to that data file and displays the current contents of this file (which is the most recent location of the GPS unit!). This is all very exciting, and it is proof that this project will actually work! Now, I am in need of some serious design and refinement. This is all pretty much dumped together, and it is in no way elegant, easy to follow, or efficient.
So, my new goals are to:
Hopefully I can talk to Dr. Pankratz about these goals in tomorrow morning's meeting with him.
Oh yeah, one more thing - to compile a socket program on Unix, you need to compile like this:
Since my completion of a working prototype last week, I have decided to concentrate my efforts on designing the "final" version of my system and documenting the design so it is easier for me and others to understand. My first step in the documentation was to create a diagram that shows all of the components of the system. Although this may get more detailed in the future, I think it provides a good overview of the system yet shows some detail as to how everything works. The overview diagram may be viewed here. I think it will also be very helpful for the walkthrough with my classmates, which will take place the week before spring break.
One issue that came up in last weeks meeting with Dr. Pankratz, Dr. McVey, and Mr. Blahnik was where I should do the actual conversion from raw latitude/longitude data to Degrees-Minutes. There are three places which this could be done - by the client application on the laptop, before the data is sent to COMPSCI, by the program that receives the data from the socket and stores it on the DAT file, or by the Perl program that is executed whenever a browser accesses it and reads the data off the file for display. My thought right now is that the data should remain in "raw" format while it is transmitted across the Internet (sockets) to the server, COMPSCI. However, the data shouldn't be converted for EVERY reload of EVERY person that is viewing the current coordinates - that seems inefficient. So, I think the best place to convert from raw data to Degrees-Minutes is by the server program that collects the data from the socket and stores it in the text file.
Another issue that came up in that meeting was that this system needs to know when any of the many connections breaks, and it needs to recover or die gracefully. So, I'll need to incorporate that into the design of generation 2.
Yet another issue that came out of that meeting was that I should now archive the version I have, so someone that works on this in the future can look at the "BASIC" system, in a very simple form, without any bells and whistles or error checking, etc. I think this is an excellent idea and was actually planning on it before it was suggested :). So, tonight I will clean the code up so it is legible and archive it away before moving on.
One more note - this weekend I drove to Minnesota and back, and I decided just for fun to take the GPS with me. I had it on the whole way over and the whole way back, and it was a lot of fun to watch the signal strength, altitude, and speed. I found it to be very accurate, and the accuracy level stayed between 16 and 30 feet the whole trip. When I was over in Bloomingon near the airport, I was even able to lock on to some WAAS signals - probably ground stations for the airport. Even when I locked on to those, however, the accuracy never was better than 14 feet.
UPDATE: I also compiled a text explanation of the system to go along with the diagram. This also includes a current list of what needs to be included in the final system design. Here is the doc.
Tonight I "cleaned up" my source code for the various parts of the prototype by removing extra code and adding some heavy documentation. I then placed this source on the web site for the world to see. Look for it under Source Code on the left-hand navigation bar.
This morning, Dr. Pankratz and I went to the Knights Telecom store and picked up a cellular phone that will act as a modem and allow our laptop to wirelessly dial in to the Internet. We also signed up for an Infinity (ITOL) Dial-Up Internet account. The phone is currently charging, but I hope to try connecting tonight or tomorrow.
This morning I spent quite a bit of time looking in to the PVT feature of the GPS Unit. This would allow us to collect MUCH more data than location, including heading and speed, altitude, estimated error, and current signal status (how many satellites we are locked on to). My problem is, when I receive a PVT packet from the unit, it has the wrong Packet ID (compared to the spec sheet). So, I may need to determine the data structure on my own. I also tried getting PVT data off of my unit with another person's source code that I found on the Internet, and that program didn't recognize the packed ID either - so I think the unit may just be giving us data in a new, undocumented format. However, I did notice that the other person's program accurately recorded all of the correct information to the log file - so, it IS getting the data somehow. My next step is to find out WHERE that program is getting the correct data and HOW it is getting it. Once I know that, I will be able to send all of that good information back to the server, which I think will be pretty cool!
Today I set up the cellular phone to act as a modem and dialed in to Infinity/ITOL using our new dial-up account! However, when I got connected I could not get the sockets to connect to COMPSCI through the firewall. I talked to Ms. Gibson a little about this, and she gave me some things to experiment with to track down the source of the issue, and I have done those and am waiting for a repsonse from her. When I am dialed in, I can telnet to Compsci, so I know there is some path there from my machine, but I cannot telnet to port 3886 (our port). Also, when I run the client on my PC, which is on the College network, I have no troubles connecting or transferring data. So, it looks to me like a firewall issue for our specific port - maybe it did not get opened up correctly or something.
Today was also my walkthrough with the rest of the CS460 Students. I thought it went well, and I got some great feedback/ideas from them about maintaining a history (maintaining two files, one for history which records every 20 positions or so and one for the most recent location), as well as eliminating the data file altogether by having the Server C program create a new HTML file every time it receives a location. Then, the HTML file wouldn't have to be dynamically generated by a Perl program whenever a web browser requests it. This may create some new file sharing issues for us, since the web server and the GPS Tracking Server will both be trying to hit the HTML file at the same time, but I think I will give it a try for simplicity's sake. Then again, if this ever needs expanding and we want to generate maps and visuals for the end user, the Perl program may be the way to go anyways. I'll keep thinking about this one...
I think we are going to try to order two external antennas for our GPS units this week - these have a 30 foot cable on them and cost about $130 apiece. This could improve our accuracy, especially when we are in a sheltered area like a car and want to have a good strong signal.
Since my last entry, quite a bit of progress has been made. I finally got the PVT working with my unit - Professor Blahink got it working right away, and I was doing the same thing he was. So, I took my unit to him, plugged it in to his computer, and my unit failed to work with his program! The unit was returning a data packet every second, but that packet had a different Packet ID than the one that we were expecting (the documented ID for PVT data). The only difference between our units that I could think of was that I had flash-loaded the newest version of software into my unit from Garmin's web site, version 2.08. Professor Blahnik had not yet upgraded his, so he was still running version 2.04. I then went out to the Internet, found version 2.04 of the firmware, loaded it in my machine, and it was working properly and returning the expected packet ID! So, I now have a program that sends the PVT start command and then reads the PVT data every second, displaying it on the screen. The data I am currently getting is altitude, fix type (number of satellites locked on to), estimated error, velocity (speed and direction), latitude, and longitude. Now, I am going to work on sending this back to the server, but I first have to resolve the socket problems I was having with connecting to COMPSCI while outside the school network. Last week I determined through experimentation and Ms. Gibson's assistance that the firewall is allowing access to our port but there seems to be a problem with my program because she can't see it listening through one of her admin tools. So, I have some sample port code from her that works for sure that I will compare to mine to see if I am missing something or doing something incorrectly.
One quick note about the PVT data - when I first got my unit to send back the data with the right packed ID (and data format), I was getting junk as data even though my structure looked just like Professor Blahnik's and the source code from another program I have been looking at as a resource. However, I remembered that C/C++ usually add padding to structures between individual data members to align them on an 8-byte boundary (or something) by default. Since the data coming across the serial port was not padded in this manner, when I did a memcpy or pointed a pointer to the incoming data, it was not lining up with the structure definition and I was getting junk. My solution was to add a #pragma pack(1) to the beginning of my structure declaration and a #pragma pack() to the end. This means everything defined between those two lines will be aligned to 1 byte, with no padding added.
Over break, the laptop company sent us a new network card for the laptop because the dongle (the cord that plugs into the card and allows you to plug in an Ethernet cable) was faulty on the original. So, I got that set up and then turned to focus on the socket problems I was having before the break. I was able to connect to my server program through the campus network but not from outside the network (for example, dialing in to ITOL). So, I compared my source code to the examples Ms. Gibson provided me with and found that I was creating my socket with the AF_INET parameter and she was using the PF_INET parameter.
When I made that change, along with a few other minor changes, I was able to succesfully run my beta client program over the wireless connection! This is HUGE progress - I now have a COMPLETE working prototype - wireless and all! My next step is to merge my PVT program into the one that succesfully uses sockets across the wireless connection. This won't be terribly difficult but I may not get to it tonight - we'll see. When I do that, I will probably also implement a protocol between the client and the server, with packet IDs or something, so we are sending more than just the raw data across and can add additional packet types in the future if we so desire. For example, we might want to just send the position, or we might want to send a whole PVT structure. With two different packet IDs, the server would know how to parse the data when it arrives based on the packet ID.
One note on the TrueSync cable that connects the Cellular Phone to the Laptop - Debby from Knight's Telecom couldn't get one, so I had my dad look into ordering one for us. When he did some research, he found that the USB version of this is for a different model phone. The cable for our phone only comes in Serial. So, I am currently using the serial cable with a USB->Serial adapter of my own, because the one serial port on the laptop is being used to connect to the GPS unit. So, the department will have to purchase one of these USB->Serial adapters. I plan on proposing this to Dr. Pankratz this week when we meet.
Tonight I merged the PVT program (which also uses lower-level windows API functions to access the serial port rather than the port class I was using in the past) with the beta version of the prototype to create version 1.0 of my system. I still have not had a chance to implement a protocol over the socket connection between the laptop and the server, but I have begun to add checks to make sure the GPS unit is still connected and responding before sending random data to the server. Also on my list of goals is to implement some sort of history to add to the real-time aspect of this, and also to make the screen the user views (the web page) a lot nicer looking. This could include a map, but I haven't looked into that much since my initial research. For the history, I am thinking I will log every 100 entries or so (maybe more) so we have some idea of where the unit has been, but are not storing EVERY entry to arrive.
I just had a meeting with DCP. He told me to buy two antennas for the GPS units and also a USB->Serial Adapter. He also recommended I write the interfaces to the GPS Unit, COM port, and Sockets using C, so they can be put into a DLL and called from C, C++, VB, or even assembly language. The major thing we discussed, though, was the addition of two-way communication to the system I already have. We decided that two-way communication over the socket connection will be fairly simple, as long as the laptop initiates the connection because its IP address is not static, but we ran into some issues when considering how to get the client on the web browser involved in this. What we would really like is for each client to have a number of commands they can send to the laptop. The client would then received some sort of feedback from the laptop indicating the result of that command. In our case, the commands would likely be different types of data, such as SEND CURRENT TIME, SEND TIME EVERY 3 SECONDS, SEND CURRENT POSITION, etc. The feedback would then be the data itself. If we could create a system that did that, we would have an excellent system that could easily be applied to a robot, model train, or something else that takes commands.
The issues we ran into mostly had to do with the web clients talking to the C program that has the socket connection to the laptop. We need to keep each web client separate so we send it the right data. When the client sends a command, they will be executing a Perl program. This Perl program needs to convey the command to the C program, which will send the command over the socket and wait for a response from the GPS Unit. This response then needs to be sent back to the client. However, I don't think we want the Perl program to wait for the response because it needs to build a page and send it back to the web browser before the web browser times out and gives up. So, I think we need to somehow store the responses on a file (or multiple files) and have the web browser auto-refresh to keep checking if its response has arrived. When the response does arrive, it can pull it off the file and display it to the user as a web page.
I think I need to talk to Ms. Gibson about uniquely identifying each client in Perl. My two thoughts on doing this right now are by using the IP address or the Session ID. If I could obtain either of these from the Perl program, I could send that unique identifier to the C program, which could then store that unique identifier with the response so the client knows which response is theirs when they come to pick it up.
Well, I have been thinking about this whole two-way communication / command-and-response thing for the last couple of days, and I think I have come up with a method that would work. This method may not be the most elegant, but I think it will work well. Here is my plan: when the end user inputs a command on a web page, a Perl program will be run which will interpret the command, obtain some unique identifier for the user (IP Address or Session ID or login name) and write that command, along with the identifier, to a text file that will act as a command queue. The server program will continuously check this file for commands, and if a command is found, it will pull it out of the file (off the bottom line of data) and will send that command to the mobile unit via the socket connection that was made by the mobile unit at startup. The mobile unit will then receive the command, perform some action based on the command, and send back some sort of response. This response could be an acknowledgement/confirmation, or it could be dynamic data, depending on the command. When the server receives the response, it will write the response, along with the client's unique identifier, to a response file. The server program will then return to checking the command queue for commands. The end user's web browser will be forced to auto-refresh every 1 second. Every time this occurs, a (different) Perl program will be executed. This Perl program will go out and check the response file for a response for that user/client. If one is found, it will display the response on the web page. If one is not found, it will inform the user that it is still waiting for the response and the browser will be forced to auto-refresh after 1 second.
One thing I think I should do at this point is convert from a text file to a binary file for the two data files that store commands and responses. Binary files will be faster and easier to treat as data structures, such as queues and arrays.
Hopefully I will get to implementing this plan this week still. I already have two-way socket communication working, so switching over to binary files and adding these features shouldn't be too difficult of a task. By Tuesday I would like to be able to click a button or link on my web browser (select a command), have that command sent to the laptop and displayed on it's screen, and have the laptop send a hard-coded response back, which will appear on the web browser. From there, it will be just a minor task to generalize it for many commands and responses.
This morning I got some of the implementation done on my new system design outlined above. I currently have the ability to select a command from the web page and have that command written to the queue along with the client's IP address. The server program monitors this queue and when it finds a command, it gets the IP address and the command and sends that command to the client (laptop), which displays that command on the screen. While doing this, I also found out how the server can tell when the client disappears or the socket connection is broken. The server will receive a SIGPIPE signal when this occurs, so I am trapping that signal, closing my end of the socket, and exiting the child process when that occurs. Now I need to generate a response at the client end and send that response back to the server. The server then needs to receive this response and write it to the responses.dat file along with the IP address of the client. I then need to write a Perl script which checks this response file for a response matching the current client's IP address.
One other thing I am having difficulty with is the handling of the command queue as a text file. Since I am writing to this in Perl and reading/deleting from it in C, I decided not to convert over to a binary file. However, I am having difficulty coming up with a method of removing the top line of a text file and shifting all lower lines up. I suppose I may have to read the entire contents of the file into an array and then write them back out to the file. Another alternative here would be to rewrite the Perl programs as C programs, which could also be run as a web page. I could then use binary files because I would have the structures and data types for them in both programs.
Today I worked on cleaning up the two-way system design I had specced out and started on last week. One of my problems was maintaining the command queue between the Perl programs that are executed when the web user clicks on a command and the server program that communicates with the mobile unit. After some frustration over how I was going to implement this using text files, I decided to do a little research into what UNIX has to offer. This led to my discover of FIFOs or Named Pipes. We may have covered these in Operating Systems, but I sure didn't remember them until I came across them in the UNIX book Dr. Pankratz loaned to me. These are essentially pipes between two programs, but they are identified by a path and file name, so any program (if it has rights) can read/write to it. Additionally, since it is a pipe, no information actually stays there permanently - it remains there from the time it is written until the time it is read, and it is deleted automatically. Also, each end (read and write) will block until there is someone reading or writing (the opposite of you) on the other end. So, I created one of these in my home directory and set up my server program to read from this pipe when it gets a socket connection from the client. This read blocks until something is written to the pipe - so as soon as someone goes to the web page and submits a command, the Perl program writes the command and the IP address to the file, which gets passed to the server program. The server program then pulls out the IP address and the command and sends the command down to the mobile unit, which generates a response based on the command that was sent. This response is sent through the socket back to the server program. I now need to determine how I will deal with these responses. Since there may be multiple web clients waiting for responses, I cannot do pipes because the data needs to remain for the next client to check. I will do some more research and see if I can find another handy UNIX structure to handle this. I must say, it is pretty cool to fire up the laptop and have it sit and wait until you click the "submit" button on a web site, and to have the command you selected show up on the laptop screen!
The last week or so, I have been contemplating how to get responses from the server program to the web client. I was considering using FIFO named pipes like I did for the commands, but this will not work because either side (read or write) will block until someone is on the other end, and I do not want to make my server program wait until the client checks for their response to continue processing. So, an idea Dr. McVey and I came up with is to have a text file of responses along with the IP address of the client they are for. When the client gets a response, instead of removing the line from the text file (which is difficult and time-consuming), it simply replaces the first couple of characters for that line with "DONE" or something. This means responses would build up on the text file and make it very large, but we could then have a purging program that goes through the file and copies all "unreceived" responses over to a new file, leaving the DONE responses behind. I think this is what I will implement for the time being, seeing as how I have not been able to come up with a better method of doing this.
This week I also documented my new command-response system design. The visual can be found here and the written description can be found here.
I just got done meeting with Dr. Pankratz, where we took a look at the overall text-based command-response system design. On a high level, we decided, the design is fairly simple and easy to abstract to many applications, which is excellent. We also decided if there were to be more than one mobile unit or laptop (like two robots), we would simply duplicate the system for the additional units instead of complicating the design by adding another variable.
One thing that was still bothering me about implementation was the method of getting responses back to the web client user from the server program. I showed Dr. Pankratz the design that Dr. McVey and I came up with last week, with a text file and a purging application that runs every couple of minutes, but that was a very inelegant and inefficient solution. So, after a little while we came up with a solution that makes so much sense I wonder how we haven't come up with it until now! The solution is this: when the server gets a respone back from the client (which will be plain text), it will create a file whose name is the IP address of the client. Then, when the client (Perl program) goes to get the response, it will simply check to see if a file exists with that IP address for a name. If the file does not exist, it will check back in a second. If the file does exist, it will read the response from the file, create a web page that displays that response, and will then delete the file. This means we don't have to worry about orphaned files or responses any more! I think I will go take a look at coding it up so we have a complete working system!
Wow, time sure flies when you're having fun! This is my first journal entry in nearly two weeks, but that doesn't mean I haven't been hard at work on my project! Since my last entry, I have completed the implementation of passing responses back to the user, and the design outlined immediately above seems to be working very well. I have also been focusing quite a bit on documentation and making this whole system easy for someone else to take over or expand upon. I would like to get to the point where, in order to add a command to the system, you would be required to touch as little of my code as possible. So far on the server end this is working very well - the Perl program simply reads a list of all commands off a text file and displays them as options on the screen. On the laptop end, I simply call a function and pass in the command as a string whenever one arrives. The user, then, must add the new command to a large IF-ELSE-THEN structure and add whatever code they would like to execute when that command is received.
Another thing I have been focusing on lately is making this system robust, so it will recover or die gracefully in the event of a connection loss at any point in the system. This was causing some problems, especially when my server program needed to watch the socket connection and the command queue at the same time (both of these reads were blocking). My solution was to make the file handles on these both non-blocking, so if there isn't anything in the command queue I can quick make sure the socket is still alive and then check the command queue again, etc. etc. I still have some more testing to do, though, to make sure the system recovers gracefully in all situations. I also need to do some testing with multiple clients - in theory, there shouldn't be any problem with two people clicking "OK" at the same time, but I just want to make sure things work properly in practice as well.
Well, I presented my project last night and everything went very well. I think people were interested in my topic and the demonstration was a success. Now, I am focusing on finishing up any documentation, especially for future students who work on this project. I have created a step-by-step guide for setting up and running the system, and I also have created a guide for adding commands to the system. Both of these may be found in the DOCUMENTATION section of this site. You may also find a web version of my Powerpoint Presentation from last night there.
