Well, this is it. This is the last blog post. A lot has happened since my most recent blog post. For one, I have completed my project. All of the functionality that I wanted to get into it is implemented. There is of course still more that I could do if I were to continue working on this project, but that will always be the case. Secondly, I had my actual presentation of my project. I thought that it went very well! My classmates were very supportive as were the faculty and I got a lot of good questions and kind words of encouragement from everyone.

I don’t have much more to say. This capstone class has probably been my favorite course I have ever taken. The opportunity to build an application (and an interesting one for that matter) from the ground up by myself was awesome. The support that I had not only from my peers but also from the incredible faculty was amazing. Special shoutout to Dr. Pankratz for his incredible Mona Lisa and his interest in my progress, to Dr. Diederich for being the one who suggested I get this project and helping me throughout, and of course to Dr. McVey for being an unending well of support and knowledge. Thanks for everything, I can’t wait to see what comes next.

The method I’m using for saving and loading settings is fully implemented at this point. I have a few more things that I need to implement into the UI such as a quick view of the current user settings in the main menu and a reworking of some of the sizes of buttons and such.

The next thing that I really need to work on is developing test applications that will evaluate the precision and accuracy of my application. This is something that I have been putting off in lieu of working more on the application itself but, seeing as we are getting pretty close to presentations at this point, I really should get some of these tests developed.

Since my last update, I have worked on the user system. I have done a lot of the foundational work with reading and writing to files. This will allow the user of this application to set defaults and save different collections of settings depending on their needs.

Today was my day to present a walkthrough of the progress I have made so far to my fellow classmates and professors. In preparation for this presentation, I added the most important feature imaginable to my project… a dark mode setting. Finally, now it may actually be usable!

I thought the walkthrough went very well. I really liked seeing what everyone else has accomplished at this point as well as showing everyone the progress I had made. I got a lot of great feedback regarding adjustments and improvements I could make to my project. I really appreciate the suggestions and I am looking forward to implementing them soon.

I met with Dr McVey, Dr. Pankratz, and Dr. Diederich today to show what I had developed so far. While they enjoyed using it for themselves and I did get a great picture of some art Dr. Pankratz created, I was unsatisfied with how the click functionality worked (or, more precisely, didn’t work). They did have notes for me regarding the UI which I will certainly have to work on, but I am prioritizing getting the click functionality to work well.

I was running into a lot of issues with getting signed out of windows when I clicked on the Windows Form. I was very confused as to why this was occurring. After looking into some documentation, I think the issue is that I was running mouse interrupts from within the form which causes issues when the CPU also has its own way of dealing with mouse interrupts. I still don’t fully understand the issue, but I found that most people found a way to avoid this issue using a thread to place the Windows Form upon. In order to accomplish this, I had to look into threads and how to implement them. I was finally able to accomplish this and now, from my preliminary testing, the clicking certainly works much better than it had. 

Since the last update, I have implemented a sensitivity adjustment module to both my code and my Windows Form application. In order to accomplish this, I had to reimplement the way I had dealt with smoothing using a class so I could more easily adjust the bin size and thus, adjust the sensitivity of the tracking. The tracking sensitivity is directly linked to the size of the bin (how sensitive the average position is to a new data point).

I also added a slider to the Windows Form so the user would be able to adjust how sensitive the tracking is on the fly. I want to work on clicking next and allow the user to set a button to click with as well as a way for them to save their settings.

After meeting with my professors, we have come to the conclusion that I will be allowed to use one button on the keyboard to accomplish clicking tasks. There are a couple of reasons for this. Firstly, it would be very difficult to incorporate clicking functionality using only the technology that is at my disposal. The first idea was to use blinking which would be ineffective as when a person blinks, their gaze moves downward (or at least that is how their eyes are tracked at least). This would make it very difficult to click accurately. Another idea to click would be to use winks. Unfortunately, not everybody is able to wink and, even if we did try to implement that, the API that I am using does not allow me to track individual eyes, only the collective gaze. The second reason that we decided to allow me to use a single button would be that there are many interfaces that allow users with motor disabilities to still press at least one button.

After we decided to allow that, I got to work on how to actually implement that into my project. I eventually figured out a way to watch for user input of a specific key on the keyboard and to click where the current position of the mouse is. There was a lot of debugging to do. However, I am now able to set a keyboard hotkey to use as a left mouse click instead of using the actual left mouse button.

After a good amount of work, I was successfully able to implement a smoothing algorithm that helps the eye tracking function stick to the user’s eye more closely. Natural sudden jitters from the eye make tracking a precise point the eye is looking at and always mapping the mouse to that position, causing the mouse to jump around and making the application very difficult to use. That is the reason that I had to implement a smoothing algorithm.

Implementing this algorithm was somewhat difficult, instead of setting the mouse to wherever you were currently looking, I kept a rolling average of the last 5 positions that the user was looking. After debugging this implementation, I immediately noticed the improved user experience. I will continue to brainstorm better algorithms that can work efficiently and consistently.

Over the last week I created my PERT diagram which will give me a basic outline for the flow of my project progress. It will give me some goals to try to meet. Also over the course of the last week I started to work with Windows Forms a bit to learn how to use it and decide if I could use it to create a UI for my project. After some experimentation, I was able to migrate my project over and get a basic UI layout that allows the user to toggle the eye tracking on or off using a button. Additionally, I have another button that opens an options menu which I have not done much work with yet. Getting this UI up and running (even though it is incredibly basic) makes me feel like I am making a lot of progress as now I have something very clear to show for my work.

We have liftoff! I have spent the last few days experimenting with the Tobii Stream Engine API and looking at the documentation online (which has been invaluable by the way, very well written and easy to understand). The engine works mostly off of an event based system so you subscribe and unsubscribe to whichever streams of information is most useful to you at the time. I have not done much work with events but I looked at some tutorials and I think I understand them pretty well now. 

Anyway, after some initial difficulty with properly configuring the libraries, I was able to get a message to pop up on screen saying my device had been recognized. That felt great, but I didn’t stop there.

I went on to subscribe to the gaze_point data stream and after some fiddling, I got the coordinates of where I was currently looking to show up on screen! Now, THAT felt awesome! Tomorrow I plan on looking into how to move the mouse programmatically. We also have our mini-poster sessions tomorrow so I am going to plan for those a bit and then call it a night.