Question Using Driver4VR with Embody and Timeline?

[Jocks3D]

I subscribe to several Meta Quest 2 channels and keep seeing posts about an app called Driver4VR that lets you do full body tracking in VR Chat with a phone or tablet. I don't use VR Chat but I do use VaM so I'm curious if it's possible to use Driver4VR with the Embody and Timeline plugins.

Driver4VR has been mentioned in passing here on the hub a few times but no one has given a full review or tutorial on how to use it in VaM. Would love to see that!

 

[Jiraiya]

If it generates SteamVR Trackers (I think it does) and you use SteamVR, the answer is a simple YES.
It's a bit of a faff to setup in VaM, but I have done full body tracking in VaM in a couple ways for quite some time now.
However, I have a personal moral objection to the way they sell their driver and require internet connection to use it. I don't mind paying, but when their server doesn't respond for whatever reason you lose you paid for driver.

So I actually use this,

Amethyst by K2VR Team

Open source VR body tracking for Kinect and other devices

k2vr.tech

It's free but requires an MS Kinect. You can grab a Kinect for a few $ now, I have three already. I would personally rather spend the money on the Kinect and use the free driver than pay for a driver I might not get to use when the internet is down or their server is offline.

Tl;Dr, Yes, it should work in VaM.

 

[Jocks3D]

I tried the Kinect option but still have trouble with the feet. The Driver4VR app seems to solve this, at least from the videos I saw. Did not realize it needs an internet connection. I wonder if someone in China or N. Korea is recording the video.

 

[Jiraiya]

The feet thing is a hardware issue, mostly with Kinect V2. They are working on a fix for it and it's dramatically improved from what it was.
They are also working on a "from scratch" AI pose estimator using the depth data only, not relying on the Kinect Pose data at all. When that is working all the many year old jankiness from the Kinect will go away at the cost of some CPU power to do the work.

 

[Jocks3D]

@Jiraiya which VR headset are you using for mocap with Kinect?

 

[Jiraiya]

I use a Quest2, Kinect V2 (because of the FoV being larger than V1) and the free Amethyst mocap software.
To be honest though, you can also use the plugin here on the hub that works direct with the Kinect and bypasses SteamVR if you don't use it.
I hope eventually all the mocap stuff lands on standards around the OSC stuff like VRChat uses, so it doesn't matter what hardware or software you use, it would all just work together.
I don't think it would be a huge problem to take the existing kinect plugin and make an OSC version that could take any trackers in the same way VRChat does, and that's it *done* as far as VaM is concerned, then you just get whatever tracking system you want that supports VRChat via OSC.

The next major thing for me is the hand/finger tracking that VaM seems to miss with my current setup. I can't get full finger tracking from the Quest2 into VaM and assume it would need a plugin to map to the finger positions in VaM. I am trying to build a pair of finger tracking gloves with work with SteamVR so that should work... I hope.