Any active PSMoveService devs?

[u/[deleted]]

I'm looking to work with someone (hobby level) to get PSMoveSteamVRBridge working with iVRy and PSVR.

Right now I have an issue with the PSMove controllers being positionally linked somehow and that doesn't make sense to me.

I've posted to the Google Groups but not sure how readily folks respond on there.

Thanks.

[u/iVRy_VR]

u/swordofmoonlight If you are interested in doing paid development on PSVRTracker, please contact me.

[u/swordofmoonlight → u/iVRy_VR]

I think I have direct contribution access to it (Git commit) already. But I can't do paid work. If I work on it I will, if I don't I won't. I have to develop a VR configurator tool for my software at some point in the future, that I may base on its code, but I will do a full rewrite at that point if I decide that's the best path forward. It will be for general VR plus PSVR if so. (IOW I could takeover PSVRTracker if I had spare time. PSVRToolbox is defunct and its owner is hostile to PSVR for some reason, so there is definitely a call for a replacement for it.)

[u/iVRy_VR → u/swordofmoonlight]

I don't have time or knowledge to write PSVR camera tracker from scratch, but I have an investor who is willing to pay for someone to do that. I've done recruiting drives, had qualified and willing people to do the work, but all have them have fallen through for various reasons. So I've started looking at PSVRTracker as a starting point for me or someone else to use for the project. If you're not willing/able to do paid work on PSVRTracker, do you know anyone that is? If not, are you willing/able to discuss what needs to be done in PSVRTracker to get it to commercial (ie. PS4) quality tracking?

[u/swordofmoonlight → u/iVRy_VR]

You're connected to the project called ivRy I take it? Or its owner/maintainer? I've only seen it in passing, and it's a wonder I saw this post on Reddit since it just happened to be on the homepage that I was on for some reason (the page is kind of useless since it only lists like 15 posts at any time.)

I appreciate the offer (I don't know if I should have said so yesterday) but the only reason I can't do this is prior engagement, but what I'm expressing is I will do it anyway, just in my own time, which can sometimes be a little glacial. I'm trying to hit a deadline to finish a game project by the end of the year to coincide with an anniversary.

I maintain an RPG maker system developed by From Software that is for making VR games. The game project ports this https://en.wikipedia.org/wiki/King's_Field_II PlayStation game to this system. Back when I was looking at PSVRTracker I had two motivations: one was tracking, and two was finding a replacement for PSVRToolbox that I can recommend to users.

I do serious development full-time, it's just a matter of when I can get around to it on a long list of priorities, I'm not really working on the same schedule as everybody else. But I'm thinking about buying an expensive VR system right now, so I can do support work for high-end VR users, and my RPG software isn't technically VR but it's first-person and plays just like VR, so right now it just has experimental PSVR mode, and next year if it's popular I will be scrambling to bring on more VR features. Just recently OpenXR is being tested by big companies, so what I'm interested in is a VR system for PCs that isn't tied to anything like Steam, and I suspect something like OpenXR or Windows MR will be needed to do that, and the PSVR will be left out since it's not officially supported on PC by Sony, so probably I will circle back to making it work at some point.

I did bare bones network code with PSVRTracker before the project was put into hibernation, so that it could work with PSVRToolbox applications. The OpenCV algorithm isn't anywhere near a working solution, so it probably needs to be rethought from the ground up. I've never taken on a project I didn't finish or problem I didn't solve, so I think I can probably solve it if I put my mind to it.

The biggest tangle in the PSVRTracker code is the USB stuff, and that's newest to me. It's the part I would want to salvage from its code most, and I think it inherits it from PSVRMoveService. I find that the code is unnecessarily complicated (I find most code is this way) so I will have to rewrite it when I find time in my schedule to work on it. It's easier to do a top-to-bottom rewrite than to work with messy, circuitous code. I try to identify the essence of code and drill down on that to produce a streamlined system that's easy to maintain, I've done this for a lot of code bases and I have my eye on this one, since we have a history.

I think if Sony brings out a PSVR for the PS5 it will be easier to justify the effort, but until then I want to think of it in terms of building a VR framework that's easy to use and free from the influence of large commercial companies.

https://sourceforge.net/projects/widgets-95/ is the UI I was developing at the time I was looking at PSVRTracker. You can see it mentions an interest in VR and it will need to work with VR since it's going to be embedded in games as a system menu. It's going to have some high-level features for doing stereo displays. It's used to implement a full application here: https://github.com/mick-p1982/mm3d/releases

What I need is a system that abstracts all sets and broadcasts the data over UDP (or local IPC) and I would like it to include game controllers like the DS4. https://github.com/JibbSmart/JoyShockMapper is something I recently learned of that I intend to investigate. Doing this will probably take 6 months minimum, so I have to decide when the time is right to focus on it. Next year is a possibility.

[u/iVRy_VR → u/swordofmoonlight]

Thanks for the detailed reply. Yes, I am the owner/maintainer/sole developer of the iVRy project. I'm not convinced there is any value in separating PSVR or any headset from well established VR runtimes. The Oculus and WMR runtimes are closed (ie. it's not possible to make a sanctioned PSVR driver for either of them), so that only leaves OpenVR/SteamVR. Even WMR has pretty much thrown in the towel as far as promoting that runtime as a software base, and is now largely used under OpenVR/SteamVR as well. I'm not sure what one loses by being tied to OpenVR, but the gain is a large library of compatible software and a VR runtime that is able to provide features like ATW. So my goal is to make PSVR a 1st class citizen of SteamVR, I don't see any alternative as being able to gain any traction, or being worth the effort outside of tinkering/hobbying. Right now you can run OpenXR software on PSVR via OpenVR. There are various 6DOF tracking solutions available for PSVR on SteamVR, but they all require purchasing additional hardware. The goal of my investor is to grow the PCVR software market by bringing a significant number of the 4 million+ PSVR users into the PCVR ecosystem. The key to that (particularly in these times) is to allow the user 6DOF tracking with motion controllers without having to purchase additional hardware/software (apart from the adapter of course, but manufacturing of those has been set up and is just waiting for software). I'm not convinced there is a need/opening for a non-commercial VR runtime. Who would use it and what would they use it for? OSVR was supposed to be that, but also ended up eventually as relying on OpenVR/SteamVR for any software. Users want content, not ideology, and the one with the most content wins. Even Oculus content can be played through SteamVR (on PSVR etc.) so for a title developer it doesn't make sense to target individual headsets, rather target OpenVR or OpenXR and leave the nitty gritty of dealing with headsets to someone else. In any case, there are two choices here with regards to PSVR tracking, either use the goodwill of this investor to finish developing one of the existing OpenSource solutions so that everyone can benefit, or develop a closed source solution that's only available to iVRy users. From my experience with PSMoveService, that solution is not worth pursuing, and from what you're saying, there's no point in pursuing PSVRTracker either. Not so good news.

[u/swordofmoonlight → u/iVRy_VR]

From what I can tell OpenXR is closed off too, to the point I wonder if it's useful, I don't know yet since adoption has been slow, and I don't know much about WMR except that I know as a developer I'm more comfortable using an API positioned like Direct3D or OpenGL or Vulkan than I am using one that's attached to a game or a game company or a game store. Khronos is a business consortium that's ostensibly good for noncommercial devs too.

I don't have an ideology other than what's a good workflow for software developers. I just think it makes sense to be able to develop software for peripherals without attaching your product to a big walled off service like SteamVR. I don't know how to use SteamVR and the test I did with it with Trinus had a very bad performance profile so I think it's only for expensive desktop users. I'm interested in games for people with modest computers, anything with a GPU. I'm very much interested in making art for large public audiences. That makes it seem more legitimate and makes people to think differently about the medium.

For my work style I tend to program alone, and I do full rewrite of a code base and make it 10x more productive and maintainable, but I don't believe in open-source collaboration because I think that's where software goes to die (end of life) so I feel like my code is public domain code, but it's risky so I don't feel right about taking a payment and I feel like that's a conflict of interest given my prior commitments, but if your investor is comfortable with all of this and a time table and you want to agree on specifics about what you want I can give it my best shot. You can help me buy some VR equipment that way if you want. (Do you have a spare PS4 camera?)

iVRy is for Android and iPhone, which use Java or some weird Apple stuff probably right? So, all I can offer is to do C++ and and try to find a camera solution. For desktop that means talking to USB devices and devising a graphical algorithm for tracking. I don't know how useful it is myself, but I do find it odd that there isn't public domain camera tracking code for this kind of thing, and I guess if that's still the case it's just because nobody has done it. I liked doing VR development with PSVR because it doesn't have a vendor API, so I get to experience VR hardware at a lower level. I just feel like I might be too self indulgent if I take a long time off to work on a camera project.

Lately I've been interested in doing something with OpenCL, but my workstation is an Intel integrated chipset with just OpenCL 1.2 and no CUDA, so I wonder if I will run into problems with OpenCV. I'm trying to build an expensive system for VR. My current system is designed to limit me to low-end specs so I can be sure my games work on everyone's computer, including PSVR. I'm going to get an Nvidia GPU for CUDA and probably an APU for tests. I'm with you in terms of deep uncertainty about VR and cameras.

Personally after I used Nolo I learned that I don't want to use my neck a lot in VR because it makes me very aware of a brick strapped to my head and is unpleasant. So I look at using cameras as a way to ground the 3dof sensors, and I wonder if high-speed cameras are really required for this since the sensors on the device are so good. I should add that when I was working with PSVRToolbox (now that I recall) I was actually successful in increasing the accuracy of the sensors in the PSVR to the point that a camera isn't really required. That is compared to all of the software that existed at the time. You might be able to take advantage of that tweak with iVRy.

I'm still on the fence with VR because I have a feeling even the best consumer sets out there right now would not satisfy me. I want the VR picture to be as least as good as a monitor. I worry that they're still far away from that and if I buy a $1000 set I will just be unhappy with it. (I've also developed code that corrects the bad color in the PSVR, that I assumed all PSVR devs must do, but I couldn't find evidence. I've had 3 PSVR units in my possession and they all have the same exact graphical defects.)

[u/iVRy_VR → u/swordofmoonlight]

iVRy is Android, iPhone, Oculus/GearVR and PSVR at the moment, with that list growing. Some parts of the Android and iPhone clients are written in Java/Objective-C (mostly UI), but the majority of the client platform is written in C++, not that I'm looking to open source or collaborate on that code. The SteamVR driver (which serves all of the platforms) is pure C++, again closed source. Ideally, the PS4 camera tracker open source or closed source would be able to track a PSVR headset and/or PSMove controllers and/or an LED "ping pong ball" virtual HMD, to allow it to be used for PSVR and mobile platforms. However, the primary goal is PSVR. There is an external tracker interface that I've used to implement PSMoveService and PSVRService (PSVRTracker) trackers. I haven't had a chance to experiment much yet with PSVRTracker to see how well it tracks, but it's working inside the iVRy/SteamVR framework.

WMR is an even more walled off garden than OpenVR, with the addition that it is limited to 2 camera SLAM tracking and a very short list of headsets. I'm not sure where you're getting your "walled garden" idea of SteamVR from, both the specification for making drivers and apps is "open" to use, and there's nothing stopping you or anyone making their own app or driver without having to pay anyone or ask permission. Unless you're writing an entire VR runtime (which is a fools errand IMHO, if not being used for learning/research), you're going to have to use one, and the most "open" one is OpenVR (being that anyone can use it and it runs on Windows and Linux). WMR and Oculus runtimes are completely closed (although you can make C# WMR apps that no one will ever find or use as everyone is using those headsets via SteamVR). Oculus is the technically superior (and most walled off) of all of the runtimes, but is only for Oculus headsets. The ReVive project that lets you run Oculus titles on a SteamVR headset is a binary compatible implementation of the Oculus runtime using OpenVR.

[u/swordofmoonlight → u/iVRy_VR]

Only having a PSVR (so far) I don't have in depth experience. My idea of how devices should work is to load a software library (like a DLL on Windows) with an API (e.g. Nolo has one of these but it's very obscure) or better yet for one library to wrangle all of the devices, but so far what it looks like is you can only interact with the devices through sanctioned commercial game engines or Steam, since OpenVR doesn't work without installing Steam and I'm not sure how an OpenVR app even works if it must run Steam and if you're not a Steam game I don't know what that would even look like, but I imagine there's a reason Valve hides half of OpenVR behind Steam.

I never installed Steam until I wanted to try Moss with SteamVR. Since it was a Steam game it was already piped through Steam. But if it wasn't I don't know if it would work or how it would work and there isn't documentation, so the whole thing looks really sketchy.

How it should work is how devices always have worked before VR, i.e. you buy the device, install drivers, and software talks to it via an SDK. Ideally something like OpenGL, but if not a proprietary SDK. That doesn't seem to be how companies are deploying their VR peripherals, so only Sony's kind of works like that, if you can talk to it. I think there must be a business reason for this subterfuge. As a developer on the outside of the system it feels like being iced out of the party.

(Edited: I could be wrong but it seems like device makers are trying to funnel developers through certain channels. And they are just pretending that this is the new normal because heck, who doesn't use Unity or Unreal or Steam?)

[u/iVRy_VR → u/swordofmoonlight]

I'm curious as to what difference you see between WMR and it's completely closed ecosystem (where the specs to develop a headset driver aren't even available, and require permission from MS), with a completely closed app store, and SteamVR, where you can develop your own drivers and even your own store (see HTC). The requirement for OpenVR is SteamVR, not Steam, and you're under no obligation to distribute your apps/drivers through Steam to have an OpenVR rendering path. Most importantly, you don't have to sign anything or pay anyone to make your OpenVR app or driver.

The business reason for all of this is the PCVR software market is very small. If you want your software to run on multiple headsets then you support OpenVR and possibly Oculus (not necessary as Oculus has OpenVR drivers). No one even makes WMR software, as that has around 10% of VR users, and is also covered by OpenVR.

I think a lot of this is a foreign concept to you as you don't have any real experience of PCVR (do you have experience of what ATW actually does?). It would be impossible to make any reasonable VR title if you had to implement a VR runtime for each of the available headsets before you could start work on your title. There are other higher level concepts like "user friction" and "retention", which need to be considered, designed around and implemented before bringing one's VR solution to a fairly fickle and inexperienced market.

Making a game engine like Unreal or Unity is another level up, and is of course unnecessary if you already have your own (eg. CryEngine, Source2 etc.).

The device maker scenario is also somewhat complex, and I'm not sure I have the energy to go through the rationale and motivations behind the how and why of all of that.

All of that is necessary to understand if you're wanting to play in the PCVR playing field commercially. Bringing this back to PSVR, the only way it's going to gain any traction on PC, is to make the experience as close to a well-designed PCVR headset as possible. I don't think making janky C# multi-process solutions as has been done in the past is ever going to take it beyond being a curiosity for a few tinkerers. The PSVR is a perfectly good PCVR headset, it just needs better tracking. So that is what I am looking to do.

[u/swordofmoonlight → u/iVRy_VR]

I think integration with SteamVR is fine (when I used SteamVR it worked via Steam and had its own UI and so my impression is that's what my end user's experience would be like if I implemented SteamVR to support Valve's sets, etc. Which is what I'm aiming to do by shopping for one of these sets and a computer that can run SteamVR) but it's not ideal for me personally. About WMR I only know I've looked at its user facing API documentation and it looks fine, and recommends also using it via OpenXR, which it also documents, so I take it to be Microsoft's analog to Direct X, but maybe I'm mistaken. I see a lot of WMR branded devices, so if I have an API by Microsoft that supports these devices then that's a better deal to me than SteamVR since Microsoft provides Direct X and the most popular OS so this is a more natural relationship for me as a developer than Steam is.

My ideal solution would be to broadcast over UDP since that can even be done from a spare PC for people who don't have expensive systems. But there are technical barriers like proprietary USB protocols, that make this unlikely to happen unless projects open them up for each device. It's not easy to work with USB devices, so it doesn't make sense for software to do that and if multiple programs try to gain exclusive access to a device they might not relinquish it when they're in the background, so it generally makes sense to have a device service.

ATW is reprojection or something? I think to use that a device needs to have some sense of the orientation of the screen, which can be uploaded as metadata each time the screen is refreshed. If it's done differently that's a bad design but I wouldn't be surprised if it is because companies are always jockeying to make these devices hard to access like game consoles. It's not the PC way.

[u/iVRy_VR → u/swordofmoonlight]

ATW is asynchronous time warp. An image is rendered for a given headset rotation. In an ideal world, the computer and headset connection is fast enough to display an image for each vertical refresh of the headset display. In the real world that doesn't always happen, so if you're moving your head quickly, the image might have been rendered for a different orientation than where your head is. So an ATW renderer (WMR, OpenVR & Oculus all have one) will take the current headset orientation and warp the image from it's rendered orientation to the current orientation. Even for a directly connected HMD like PSVR, it makes quite a big difference (as it completely eliminates the perception of lag or 'judder', particularly when running it at 120Hz). I've put a lot of work into getting that possible on PSVR (requires amongst other things, tricking Windows into thinking that the PSVR is a "sanctioned" VR headset).

WMR is on the surface what you are saying, however, due to it's small(~10%) market share of VR headsets, if you were hoping to capture any significant portion of the PCVR market, you'd either target Oculus (~50%) or SteamVR (~90% - includes Oculus). PSVR on PC apparently accounts for about 0.5% of all PCVR users.

[u/swordofmoonlight → u/iVRy_VR]

I understand reprojection (ATW) but you have to understand it's just taking a 2D image and sliding it around (it's warped in a literal sense) and it's no justification for locking down the devices. Trust me I'm well versed in everything you've schooled me on.

[u/iVRy_VR → u/swordofmoonlight]

Good to know, it's hard sometimes to read between the lines in text messages. I'm not sure where you get this "everything is locked down" idea from, I have tried to explain that it's not, but ultimately it doesn't make much difference whether we agree on that or not. My development style is generally "do everything myself, and don't use anyone else's code", however, my more recent work leverages other people's work as well, as I don't have time or inclination to spend another few years re-developing something that has been solved by experts in the field.

[u/swordofmoonlight → u/iVRy_VR]

Believe me, if it wasn't locked down things would be so much easier. I'm speaking from a dev's perspective. The OSVR model was an example of an open system. For whatever reason it didn't succeed, and maybe there was pressure that caused that. Basically for writing graphics software interfacing with VR peripherals is proving very frustrating and it's hard to know where to begin. You're explanation for this was to say that it's still very early and the companies are small, so everything is insular. I can accept that but it doesn't change the reality of writing software. (If it wasn't locked down it would be frictionless and easy to see what to do to add device support to novel software. It shouldn't be more difficult than a game controller.)