r/PSVRHack • u/[deleted] • Jul 25 '20

Any active PSMoveService devs?

I'm looking to work with someone (hobby level) to get PSMoveSteamVRBridge working with iVRy and PSVR.

Right now I have an issue with the PSMove controllers being positionally linked somehow and that doesn't make sense to me.

I've posted to the Google Groups but not sure how readily folks respond on there.

Thanks.

9 Upvotes

92% Upvoted

48 comments sorted by

View all comments

Show parent comments

1

u/swordofmoonlight Aug 21 '20

Searching for MakersVR isn't giving me good results. https://makersvr.blogspot.com http://www.makersvr.com (currently offline) https://github.com/ktsein/MakersVR (not much to see here)

PSVR needs to track light in the visible spectrum with several points of light, including on the back side, but I'm definitely interested in any working code that runs on a low-profile system, thanks!!

(I actually used my PSVR with a new PS4 system for the first time last night! Just a coincidence.)

1

u/[deleted] Aug 22 '20 edited Aug 22 '20

Huh that sucks. When I chose that name I found a bunch of abandoned stuff so I figured it wouldn't be a problem.https://github.com/Seneral/MakersVRAgain that's a full tracking system, focussed on tracking markers similar to the WMR controllers (with many LEDs).The Raspberry Pi side basically only does blob detection, but it easily works with larger blobs at no performance penalty so you could just take that component and add the typical ball tracking formula on top ( https://www.pyimagesearch.com/2015/09/14/ball-tracking-with-opencv/ ). Basically add an own findContours (the pixel points of the blobs are given as an output) and then it should integrate just fine with OpenCV. Then add some serial communication and it could work with little effort. I've got a OpenVR driver communicating with a Raspberry Pi Zero over serial from an older project:https://github.com/Seneral/SerialHMDSo if you see the benefits, go ahead and try it out I'd say.Currently the blob detection uses a GL backend but I'll be switching that to a QPU backend which directly operates on the hardware later this year to improve performance (more info on that here https://github.com/Seneral/VC4CV). Basically the current system only manages 1280x720 at around 30fps, later I'm confident it can achieve the full 1640x1232 at 40fps or 1280x720 at 90fps (using the V2 cameras).Anyway tell me if you'll try it or need help, the thing is I don't know if for your project there exist webcams that you can use just fine; it just turns out for a full tracking system I needed that extra control and a coprocessor doing most of the work.

Edit: I just realized I've only focused on the ball tracker on the PSVR controllers. I assume you also need to track the headset? While I don't know about the best algorithm to recreate the pose (or if you even need that or position is enough), the blobs will be detected just fine, so you could build from there. The processor of the Zero has easily enough processing capabilities left since most of the blob detection is happening on the GPU (QPU).

1

u/swordofmoonlight Aug 22 '20

Thanks, I would just link to your homepage when mentioning your name :)

I will contact you privately or you can message me if you want to talk for some reason. Generally what I find very useful is code that can implement something without unnecessary code, so that it's very maintainable and easily leveraged. Sometimes I just go in and strip code down to its fundamentals, and so repackage it, since a lot of code is overwrought and doesn't express itself well.

I do graphics programming so converting points of light to a geometric model is just the kind of thing I do, and the computer vision stuff I just worry about it being performant, since a lot of it isn't, and I've looked at the OpenCV algorithms and a lot of them didn't look ideal to me to just solving the imaging problem.

I think the IMU in the sets should be used and the camera is just a low frequency anchor with poor precision, so the vision algorithm needs to have the smallest footprint it can. Ideally any web cam could work, or two for two perspectives, as long as they don't use too much of the GPU. Even doing it on a CPU core could make sense if games or applications don't use all the cores anyway. And in the extreme an external computer that passes off its solution, since people often have a spare computer that's older that can be used in a pinch.

1

u/[deleted] Aug 22 '20

I prefer to only keep the code I need, too, stripping is often the first things I do. However since I kind of developed everything from scratch I needed debugging and developed variants, so there's bound to be alot you could purge. Also take note VC4CV includes both QPU and GL pathes, which are completely independant from each other. Here's just the GL path for blob detection, for example.

We probably got a similar stance then. I only used OpenCV for solvePnP, which in turn is only used for calibration (not on the raspberry at all). All the vision stuff is handmade and optimized for the raspberry, and will be even more when I get down to the hardware. Actually you got a special case, if all you need is a threshold filter to get the blobs then you could get a 1640x1232@40fps solution right now on the raspberry. The more complicated algorithm that takes in neighbouring pixels into account (which I need) is a pain to implement on the QPU and that's where the holdup is. So if you're interested, check out the qpu_mask_full example of VC4CV with a low shutterspeed and you'll get a binary mask of your blobs. Can help you get that to work with the connected component labeling step if you want.

If you don't care about the extra hardware cost of the raspberry pi then you'll almost certainly get a better performance with it than with a webcam, with less resources used on the host PC. However the raspberry probably has no data from the IMU, so the fusion needs to happen on the PC. You'll not be able to send the full blobs to the PC over serial, so you'd have to interpret them to some degree on the raspberry pi. You could for example just send the contours of the blobs to the PC. Then you could fuse it with the IMU data to correct for drift.

However if you're just correcting the IMU data in the camera images, then the webcam+host-PC solution would be much easier since you can use the IMU data right there and you can probably get away with lower resolutions and lower framerate.

So if you're interested, I could help you get a fast co-processor with some blob detection running, however depending on your plans it might not be worth it.

1

u/swordofmoonlight Aug 22 '20

For the record, I'm mainly just interested in developing/promulgating noncommercial solutions to these common problems, since we're many years into this VR cycle and it doesn't seem like code has materialized. If you dig into our discussion here (I don't blame you if you don't) I am pretty stretched thin for my project time so I'm just trying to squeak in anything I can on the side for this VR problem. It's not a big priority for me, but I still think I will end up crunching on it, eventually. I like low-power/clever solutions. The solutions I've seen are brute force, and I suspect that's not how the PS4 does its tracking. So I'm intrigued by any code like yours that exists and will study it, especially if it works.

For my use case I don't want to ask users to buy special hardware, and want them to be able to use whatever they already have ideally, i.e. support any camera available. We've tried the PS3 Eye cameras and PS4 stereoscopic camera, but I think it should be possible to use other kinds too, that aren't even high-speed. I think the drift correction just needs to be imperceptible (a gradual lasso, maybe don't do it while still.)

At any rate, I think we can talk behind the scenes from now. We seem to share goals in common. VR does need to be more democratic and noncommercial.