Moses Turner
February 24, 2023
Share this post:
Reading time:
In our previous blog post, I teased our new optical hand tracking pipeline, codenamed Mercury. Work on this tracking method began around January 2022.
Now, after a little over a year of development, it's finally ready for the public to use!
Mercury currently works on Valve Index, Windows Mixed Reality headsets (Reverb G2 v2 shown below), Oculus Rift S (early support), and Luxonis cameras (video shows a OAK-D Pro W Dev mounted on a North Star). It estimates hand pose accurately, supports limited hand-over-hand interactions, tracks fast hand movements, and is useable for drawing, typing and UI interaction.
Usage documentation is here if you want to run it in Monado. Expect in the coming few weeks some improvements to hand tracked-state stability, reductions to background jitter, a SteamVR driver so you can use it on Windows, and a much more detailed blog post showcasing the work we did to get here!
20/12/2024
The Rockchip RK3588 upstream support has progressed a lot over the last few years. As 2024 comes to a close, it is a great time to have…
09/12/2024
Collabora will be at NeurIPs this week to dive into the latest academic findings in machine learning and research advancements that are…
05/12/2024
Now based on Debian Bookworm, Apertis is a collaborative OS platform that includes an operating system, but also tools and cloud services…
Whether writing a line of code or shaping a longer-term strategic software development plan, we'll help you navigate the ever-evolving world of Open Source.
한국어 버전의 Collabora.com 보기
Acesse Collabora.com em Português
Comments (8)
Eli:
Mar 01, 2023 at 12:56 AM
How do I get it on windows 10?
Reply to this comment
Reply to this comment
Roman:
Mar 01, 2023 at 05:33 AM
This is very exciting! I hope SteamVR driver is coming that would let it be used in the similar way as UltraLeap does it, only it has so much better potential, as no extra hardware is needed, and HMDs cameras are wide-angle and perfectly aligned. Whe SteamVR driver is ready, would it work with existing WMR OpenXR runtime, or would it replace it? Or would it only work with StemVR OpenXR runtime (which is inferior for Reverb G2 and other native WMR HMDs)?
Reply to this comment
Reply to this comment
RalfMarco:
Mar 01, 2023 at 10:17 AM
This is fantastic!
A openXR toolkit driver would be amazing. For MSFS Community and VR Pilots. This Handtracking in an Airplane - Game changer! Many fly with the reverb G2.
Reply to this comment
Reply to this comment
hyperdrive:
Mar 04, 2023 at 11:03 PM
will it work while you are holding ANY type of controllers, so you can have access to thumb sticks and buttons while still having full finger tracking? otherwise, it's kind of useless because I shouldn't have to give up certain controller necessities for a small but wanted/fun gimic that most games don't even support. I say this because the quest 2 has this issue, and time and time again I see people on the quest 2 try to do simple things like walking and jumping while using the full finger tracking and I see that as a big problem and should have never happened in the first place.
Reply to this comment
Reply to this comment
Igor2402:
May 25, 2023 at 05:10 PM
Can you make a tutorial...
How can i use this in Rift S, unfortunately i am layman in this things...
But i really wanna use this feature. Please, make a Tutorial Video for windows 10
Thanks.
Reply to this comment
Reply to this comment
Matt Geders:
Sep 07, 2023 at 09:05 PM
Any update on the development for Windows 10/SteamVR?
Reply to this comment
Reply to this comment
Jakob Bornecrantz:
Sep 08, 2023 at 02:45 PM
You might be able to compile it yourself, changes will probably needed. But there are no ready made installable solution, we have no immediate plans to make one. Hopefully somebody in the community can step up and do that.
Reply to this comment
Reply to this comment
john:
May 05, 2024 at 10:21 PM
where can i get it
Reply to this comment
Reply to this comment
Add a Comment