The latest releases of OpenTrack now support Tobii natively, I recommend going that route now instead of this method.
Thanks to user Henkeman in the comments, I could massively simplify the IP address settings and content.
I made another small update recently to this tutorial. I used to suggest using two separate bind for centering FaceTrackNoIR and OpenTrack. However, after more experience using this setup, I've found it's actually more valuable for them to be bound to the same joystick button or keyboard shortcut. This can help with extended play sessions when your position eventually starts to shift a bit relative to the Tobii Eye Tracker.
I made small update to this document through more experimentation. You can see the recommended changes by clicking the link above. TL;DR: Using a smoothing value of 5-10 in FaceTrackNoIR can help significantly in games like DCS where there's no built in camera movement smoothing. This is different from the OpenTrack Hamilton Filter in that it can make the camera move more smoothly in game.
With the recent-ish addition of the Hamilton filter in OpenTrack, I've been quite keen to try it out with the Tobii Eye Tracker 5 that I have. Unfortunately, while some foundational code was added a while back for Tobii, not much progress has been made since then and it appears that the initial contributor for this work is no longer interested or has the time to continue the work as it's been over a year since any additional progress has been made. Because of this, I started digging around and figured out there's actually a way to use Tobii Eye Tracker 5 with OpenTrack today, it just requires an intermediary connection with FaceTrackNoIR since they already have a plugin that support Tobii 5.
One caveat of this method, eye tracking is not usable within OpenTrack, only head tracking.
Table of contents:
I'm going to break down this tutorial into a few different parts, but to start with, we'll need to install all the required apps and dependencies.
I'm going to assume you've already installed and setup your Tobii Eye Tracker 5. You will NOT need the Tobii Game Hub app, and while you can have it installed in general, I would recommend you don't have it running while using Tobii with OpenTrack and FaceTrackNoIR, or you could get into some weird problems (make sure you close it from the system tray before continuing). You WILL need to have Tobii Experience enabled and running in your system tray, as without this the Tobii will not work with FaceTrackNoIR.
For starters you're going to need to download and install the following dependencies:
- OpenTrack
- I recommend the latest version, however note that as new releases come out, some of the screenshots in here may end up differing a bit
- FaceTrackNoIR
- Pretty sure it might cost money? Not sure if there's a free version
- FaceTrackNoIR Tobii Plugin
- You can download the plugin if you click the Tobii Plugin image on the linked page above
Your first step is going to be configuring FaceTrackNoIR with your Tobii Eye Tracker 5. The goal will be to configure it as a pass through to send the (mostly) raw Tobii input data over to OpenTrack. We also want to ensure we have no curves or filters applied since we only want OpenTrack to manage all filters and curves.
To get started we'll want to match FaceTrackNoIR's settings to this screenshot,
if you've used FaceTrackNoIR before, I would recommend creating a new .ini
profile like I did called default-opentrack.ini (the name doesn't actually
matter though):
Update Sept 18, 2022: I originally recommended that smoothing be set to 1, however after testing other games like DCS, I found that having a smoothing of 5 to 10 helps considerably because unlike IL2, there's no setting to smooth out camera movements so you can get a movement that feels like it's at a lower framerate than it should be.
Ensure smoothing is set to around 5 and all invert checkboxes are unchecked.
Set Tracker Source (1st = Master) to Tobii5 Tracker. The settings for it
should look something like this:
You probably won't need to change anything in here since the defaults should be correct. The values you see for hardware status may differ a bit, what's most important is that you match the selected checkboxes.
Back on the main settings screen, ensure Filter is set to None - remember,
we only want to pass on the raw data to OpenTrack.
And finally for Game protocol, set it to FaceTrackNoIR, and pop open the
Settings dialogue for it:
Jan 19, 2023 Update: I used to have a much
more detailed section here around setting a local network IP address, but using
127.0.0.1 works just the same and requires much less setup work.
Set IP-address remote PC to the values above: 127.0.0.1. I set port to
4242, although this one is a bit up to you (and might depend on things like
your firewall settings), but either way you need to ensure that you remember
what this port is for later. If you're not sure, just use 4242.
Next up, we're going to click the Curves button on the main Tobii window.
Assuming you've freshly installed FaceTrackNoIR or created a new profile, there
shouldn't be much to do here. Either way though, if there are any curves
defined here, you need to undo them. Basically we are configuring
FaceTrackNoIR to forward on its raw inputs to OpenTrack, and we don't want them
edited in any way, as close to raw as possible. I've attached 2 screenshots
here of what my Curves window tabs look like, so double check they match:
And finally the last thing you can do, although it is probably optional, I
would recommend it, bind a Center reset hotkey for FaceTrackNoIR. You can do
that under the Shortkeys button from the main FaceTrackNoIR screen.
PLEASE NOTE: we are going to treat this as a SEPARATE centering
functionality from OpenTrack (which we will setup another bind for later,
that's more for use in game). The only real use for this centering shortcut is
when you start up FaceTrackNoIR before OpenTrack, it's generally a good idea to
center FaceTrackNoIR once before starting up OpenTrack. My Shortkeys setup
looks like this, but it's really about personal preference:
Oct 12, 2022 Update: I used to suggest you have two different centering binds -- one for FaceTrackNoIR and one for OpenTrack. However, with more experience in this using this setup, I've found it more valuable to make sure you bind Center the same in both OpenTrack and FaceTrackNoIR. This will ensure when centering your view in game, they both get reset together. Personally I have this bound to a button on my joystick.
Outside of the shortcuts, it's probably a good idea to make sure everything
else here is disabled, and having No Beep enabled, unless for some reason you
like that.
Ok, now we should have FaceTrackNoIR fully configured, we should conduct a
quick test by clicking Start:
With FaceTrackNoIR running, you should see the UI on the left reacting to your
head movements, so if everything looks good, you can go ahead and click Stop
for now and we can continue on to configuring OpenTrack.
If you've used OpenTrack before, I'd recommend creating a new profile for this, so as to not lose old settings. With OpenTrack launched, you're going to want to match your general configuration to this screenshot:
For starters, we need to set Input to UDP over network. Once that's set,
click the tool icon next to the drop down to configure it. Remember that port
we setup from earlier with FaceTrackNoIR? Now you need to recall that port and
input it into the UDP over network settings, I'd recommend leaving all the
other values at 0.
Next up, we're gonna click the big Options button on the main screen and go
through a couple tabs of settings there.
First up is the Shortcuts tab. I'll show a screenshot of my settings here
and then go over the settings I have below:
There are few required and optional settings here. I'd definitely recommend
setting up Center and Toggle binds.
This is the Centering bind you're going to want to use in game, not the one
from FaceTrackNoIR.
Oct 12, 2022 Update: You want to make sure that whatever bind you used for centering in FaceTrackNoIR is also the same bind here in OpenTrack. This way in extended play sessions they get reset together. I personally use the same button on my joystick to re-center both in OpenTrack and FaceTrackNoIR.
Basically you'll want to use this to re-center your view when it inevitably
gets a bit out of sync. Toggle is obviously a way to pause or resume head
tracking. (It should be noted that I have experienced some bugs in OpenTrack
where these binds appear to get stuck in
game, and requires
multiple pressed to get unstuck. Until OpenTrack resolves this, there's not
much you can do).
Two required settings here are the centering options. Make sure Center at startup is on, and choose Point as your centering method, otherwise you'll
probably experience your head position constantly sliding one direction or
another.
Next up is the Output tab. First here's a screenshot of my settings and I'll
outline details of it after:
For some reason I've found in practice that the X and Roll axis are inverted,
and had to invert them here. Technically you could invert them in
FaceTrackNoIR instead, however I chose to do it here so as to keep
FaceTrackNoIR settings as pure as possible.
The Custom center pose is 100% personal preference. I'd recommend leaving
everything at 0 to start with, and only change these values if you aren't
happy with where your head sits when you Center your view in game. These
values will probably need to be different for different games, if you ever do
decide to change them.
Finally we have the Relative translation tab. As I've done before, here's a
screenshot of my settings, and then I'll explain my choices after:
Pretty much this entire tab is down to personal preference. One thing that's really bothered me about the general Tobii Game Hub app, is that in cases where you need to look behind you, your side to side movements get inverted, which doesn't feel intuitive. These settings here appear to fix that for me and I've been quite happy with them.
Neck displacement is also down to personal preference, but it basically means
that your view moves a bit to the left and right when you rotate your head --
because your eyes are offset from the origin of the movement.
Now that all this is done, we should test to make sure it all works properly.
To do this, first click Start in FaceTrackNoIR, use the special hotkey you
mapped in FaceTrackNoIR to center the view, then in OpenTrack click the big
Start button. If you've done everything correctly, Congratulations! you
should see the octopus rotating with your head movements, meaning you now have
your Tobii Eye Tracker 5 working with OpenTrack!
If you're well versed in how to make your own mappings or filtering settings, you can probably skip the rest of this document. However, if you want some tips on how I tuned this setup, feel free to keep reading! At the very least, I'd recommend checking out my section on the Hamilton Filter, it definitely was a game changer for me and what got me interested in getting Tobbi working with OpenTrack.
The next big step in this process will be to setup good curves or mappings for your head movements. This is also 100% about personal preference, so while I will show a few screenshots of my curves/mappings, they may not be useful to your specific situation. The main goal is describing my process for how I landed on the settings that I have. It does take a bit of work, but I think the payoff is worth it.
A good way to think about my process here: we first want to tune high level overall range of movement, THEN tune our filters/smoothing/curves.
The first step in this process is disabling any filtering/smoothing. It's much easier to judge our ranges or amplification of movement when we are dealing with one axis of movement at a time.
It's also worth noting, that different games may require different configurations, so pick one to start with, get it working properly, then clone the OpenTrack configuration and tune as necessary through a similar process.
Assuming you've got a game in mind, the best course of action will be to launch that game, ensure it's in a borderless windowed mode (to make alt tabbing between the game and OpenTrack easy), and also that you're in some sort of parked free mission where you can test out these ranges of motion and tune as necessary without any interruptions. For IL2, I just launched a free fly mission that started parked with the engine off.
As mentioned earlier, first step should be to completely disable any filters,
if you followed my tutorial up until now, I had you select the Hamilton filter,
so for now, just click the drop down and select the empty filter.
The next prep step is to disable all axis except for one. Basically you want
to check these axis ranges one by one to figure out what feels right. You can
do that by going back into the Options screen and disabling all axis except
for one like so:
Now, start up FaceTrackNoIR and OpenTrack to enable head tracking, click the
Mapping button and select the axis tab you want to tweak. Most mapping tabs
will have 1 or 2 drop down settings to select from, usually labelled input and
output. Essentially these are high level mappings of input movement (how much
your head moves) that then gets interpolated with output movement (how much
your in game head responds). Generally I like to select the highest value in
both, to get a 1:1 movement mapping, that then I can control with points how
much this ratio should actually apply. Again, this is all personal preference
and you should play around with the points to find what works.
I generally prefer NOT to use curves -- I like a consistency to my head movements, but that's just me.
So for this first example, assuming I'm tuning Yaw, I'll disable all other
axis, have OpenTrack/FaceTrackNoIR running and sitting in game. I'll use my
OpenTrack center bind to fix my position, then in the mapping tab, set Max input to whatever the maximum value is, then make sure I have a long straight
diagonal line from corner to corner like so:
Right off the bat when you do this, you'll probably find that there will be
very little yaw rotation in game, so next up would be to start pushing the end
value left, to essentially up the sensitivity like so:
The good news, as you manipulate these points, they will effect the game immediately, so you can easily slide that point around, alt tab back into the game, test movement, and alt tab back again. Basically your goal here will be to find a good compromise for looking behind you without requiring too much head movement that it's painful or uncomfortable. Note, these movements will probably feel very wobbly and shake-y, but don't worry about that now, you're really just trying to find good range limits.
Continue this process for every axis one by one. One thing I did, was after tuning both Yaw and Pitch, I then turned both on together, and tested them together just to be sure.
Similar with positional movement -- first tune them individually, then do a pass with them on together. I left roll for last, some people don't like roll at all though, so that ones pretty optional.
Once you have all the axis tuned, turn them all on and do some tests with them on together, does it at least feel good from a head movement perspective? It's going to be quite shake-y though, so try not to let that get in the way.
And finally here are screenshots of my mappings for each axis as reference:
Just remember, these are totally a preference thing, I mainly share them here in case you're struggling to find settings you like as they could be a place to start from. They are also something that I still tune from time to time, so it's possible if I screen shotted these values today, they might differ from when I originally wrote this post.
Ok, so hopefully after all this work, setting up the Hamilton filter is the payoff you've been looking for. At least I can say for myself, it's the only way I've been able to get Tobii Eye Tracker 5 to a stable enough experience for flight sims.
Hamilton filter was recently added to OpenTrack, and oddly enough, there's been very little written or documented about it. The closest thing I've found is this page in Russian from the developer who created it that documents how it works. If you translate it into English it should be mostly understandable; I do recommend a quick read through as it will give you a bit more context (it's not long, but quite technical).
Here are my current settings for reference, and I'll talk a bit more about them below:
My overall goal in my tuning process with Hamilton, is that I can quickly look around, stop moving and have a nice stable view. If your camera is often shifting from subtle head movements, it's incredibly hard to accurately spot aircraft or maintain situational awareness while flying.
Similar disclaimer from earlier, I do often make small tweaks to these as I'm playing if I find issues, so consider them a perpetual work in process.
A quick summary of how Hamilton works -- it has two main smoothing configurations, one for head movement and one for head rotations. Similar to the process we applied above for determining your ideal mappings, you'll want to do this in game tuning with just rotations enabled or just positions enabled to get the settings you like.
There's a third configuration called Zoom smoothing, which is a neat feature
where if you lean in towards the monitor, rotational smoothing will go up and
it's much easier to maintain stability and make slight corrections to your view
while zoomed in. I highly recommend playing with last, once you've dialed in
the other settings.
Unlike other filters, I find very high values of Smoothing to be quite
acceptable with Hamilton. You'll have to play around with it in relation to
Max Distance because both values are dependent on each other. One simple way
to think about it Max Distance is that the higher the value, the less wobble
you get when swinging your head through big rotations quickly.
The Dead zone filter can be thought of as a dynamic deadzone. When you move
your head to a new position and hold, it zeroes in on that and stabilizes your
view. This is extremely helpful when you make quick changes and then stop your
movement. It can have some interesting side effects, in that often you'll feel
like you'll have to over-compensate a bit to move again, it almost creates a
feeling like your head is more of a joystick pushing a camera around, instead
of a system that's tracking your direct head movements. Given that most image
based head trackers have inherent stability problems, I think this trade off is
quite acceptable and worth the effort.
Hopefully you've found this document helpful as a way to setup and configure a Tobii Eye Tracker 5 with OpenTrack. Another nice benefit of this setup is that it will work with pretty much any flight sim out of the box given OpenTrack's great compatibility.
Feel free to leave any comments below if you see any issues or problems with this document, I can easily update it as necessary.
I probably should have indicated I was running Linux. No go for me - Tobii SDK is still the blocker. Awesome that opentrack now supports Tobii on windows, though!