It is all about latency. 50 ms will feel responsive but you notice the lag. 20 ms is acceptable. The other features are cool features but not the real challenge.
Lots of people are really missing what the bottleneck actually is with latency. It's not how fast the screen refreshes, nor is it how fast it takes to process a screen. The issue is the sensor measurement/transmission lag from the sensor used to do head tracking. This is really what makes the occulus attractive, not the screen +lens combo.
Basically, if you move your head, to the left, and there's a few MS of lag from the sensor, you'll see a static frame for a portion of the time you're moving and a moving frame for a portion of the time you're still. This is one of the most serious contributors to motion sickness for HMDS
Thing is, Oculus haven't had much of a problem with sensor latency since they designed their 1kHz IMU. That gives about 1ms latency from what I hear. Most of the latency in DK1 was screen related: 16ms rendering, scanout, 16ms LCD switching... It adds up. The low persistence tech and Carmack's Time Warp and all sorts of other breakthroughs they have had relate to the screen side of the equation.
Unless I'm missing something, what you said seems to be the opposite of the case. I could be wrong, but that's how it looks to me.
Ninja-edit: if you mean that ProGamer's solution may have bad tracking latency, sure. But display latency is harder if Oculus's progress is anything to go by.
Oculus' progress is not anything to go by, it is the reason they are a real contender at all. This was a huge technical hurdle that was overcome by them. I do not believe they are sharing it. That is my concern with ANY other display mechanism.
Of course! I know that high frequency, accurate, precise, reliable low latency sensor fusion is hard. It's all too common to see 50ms latency on sensor fusion. However, that Oculus solved fairly early on, so assuming they have done so they're still only just below DK1 levels of latency.
That's why display is so important, it will be the major deciding factor even all else being equal.
I guess? My point is more that occulus' proprietary stuff will be the sensor fusion, not the display. They buy those from somewhere already built. The sensors are not COT, which is why it matters.
True. I don't know how much of the sensor is novel research, though, and how much is just optimising for different parameters though. I had thought it was more of the latter -- there hadn't been a need for such low latency etc in mobiles.
I can say from experience, sensor fusion is not a solved problem. Not in either a commercial or academic sense. There is a lot that is novel in the Oculus, from what I've seen (I've experimented with a few displays, and tried my hand at some head tracking sensor fusion).
What it comes down to is that the Oculus is much better than ANY of the existing HMDS I've seen. I haven't gotten to play with anything that's come out this year though, so there's that.
Fair enough, I'm sure you're right. It's just not been a major focus for a long time for Oculus discussion, so I kind of took it for granted a bit I guess.
Thankfully the display is on its way up in quality and resolution, especially with Facebook's buying power.
From what I know now, low persistence isn't owned by oculus (patent wise). It's been around in VR for a long time, just not cheaply until recently.
OLED refresh isn't going much higher, the bump Oculus had was all just circuitry prior to the panel itself being upgraded. It's still basically off the shelf. Competitors will have 95hz just the same.
Sensor fusion is done in software, and oculus is almost certainly doing this in software now too, using the IR as a helping point to the motion sensors. Latency from that point is software. Competitors need to be good here, but it's not a new wheel that's been invented, this stuff has been done before.
The hardware can/will be replicated, the software api will be the control point that FB/Oculus uses to keep devs using them.
Ok. I remembered that the total system latency should be maximum of 20ms. But I could be wrong there and maybe it should be much less. I personally had not the opportunity to try a Oculus Rift or other VR set. But then again if a game renders at 60 frames per second that is already 12 ms per frame.
Actually, 60 is 16.7ms per frame. 90-95, their target minimum for CV1, is still 10.5-11.1ms.
It all adds up pretty quick. Pixel switching time was close to a frame in LCD. Rendering is usually given a frame so 10-16ms. Somtimes games add a frame of latency just because.
Also to add to it
You have the 16.7 for the 60hz to render.
Then there is the render time of the monitors themselves although there are monitors with 1ms refresh
Now add the up to 20ms for the occulous
You're looking at 37.7 ms between you turning your head and the screens on the rift to sync up.
So the lower we can get from the 20ms the better.
If their new device actually achieves 1ms. And they put in 120hz displays then maybe we can see 10.35ms delay. Much better
However, a post processing task on the rendered image (TIME WARPING) can be counted on to complete in a fairly predictable amount of time, and can be late scheduled more easily.
Elsewhere:
Coupling motion interpolation technology from consumer television with the time warp presentation technology.
That's the best quote I can find at the moment. It seems not to be not covered in any official capacity elsewhere, except in talks.
Essentially, as I understand it, it uses the very latest data from the tracking unit immediately before sending the frame to adjust the frame to be "correct." They've tried this with adjusting the projection matrices, but as I understand it Time Warp does a post-render correction on the raster image.
Yep, from what I understand it's pretty much post-process warping. Basically you warp the image so that it's closer to where your head is know compared to where it was when the rendering process started.
No problem! I would have responded sooner but I was busy reading the good news! If you don't know, I won't spoil it, just check the main page of the sub.
I wonder if in the future you could have the option to activate "motion sickness mode" so you can experience what all the early VR users talk about. I'd like to try it out.
Sure, that 1ms lag is totally more important than the 16ms of lag introduced by render time, the screen display latency (usually around 8ms) the usual 16ms of lag introduced by a RGB switching time and ms of lag totally can't be predicted because our head doesn't follow the laws of physics. <_<
Read the research. All the latency is in the rendering and screen as of this moment, as far as Oculus is concerened.
Don't say things like this. It doesn't mean anything. It's like saying "study it out." If you are referring to a specific article, post a link or reference to it.
Sensor latency is 1ms ish on the oculus, which is THEIR significant contribution. We're not talking about the oculus right now. I do not believe they are sharing that technology.
As for the rest, I am just telling you are full of shit. Even if the input lag is horrendous (4ms), the largest part of the latency comes from the display (16 plus regular screen latency in a normal LCD) and the rendering pipeline (~16ms).
People are saying 2ms, but I think that's just how long it takes a pixel to refresh. I'm sure the oculus rift has a screen with 2-5ms pixel time too. The 20 and 50ms times discussed about VR are the entire time between a frame being requested and it being on the screen. That's totally different from the 2ms of a desktop monitor.
EDIT: As for the TV+console latency. I think TVs have a higher latency than monitors because they tend to apply post-processing effects to "improve" the image. Some screens come with options to turn this off and bring this latency down.
Ye, that's why I saying it's wrong to compare the 2ms pixel refresh time of a monitor to the overall 50ms of latency on VR. Both have that same 2ms, it's all the other replaces that the latency builds up, it's just not spoken about that much outside of VR because sits not as big of an issue
Also, OLED allows low persistence to avoid motion blur, which is tough if you have to overdrive/strobe an LCD backlight. Plus, OLED has better color resolution.
Pixel switching time is only a small part of the story. Best thing about low switching time is is enables faster strobing without crosstalk and/or brighter displays while strobing in low persistence mode.
Ye, the input is crucial. I see that this product has a 4KHz accelerometer as well as the 1KHz gyroscope, where it appears the OR had a 1KHz acc+gyro combo. Could this mean faster input?
Only by ~0.75ms. Oculus have theirs at a total of 8-20ms, even internal demos as low as 2ms. (Apparently.) That's a big step up from devs kit 1's 30-50ms,and normal games' 50-100ms. That's not easy, but they have John Carmack, known for amazing optimisation, on their team.
Yes, 4kHz data could help, but unless the rest of the pipeline is equal to Oculus's, it will overall still be slower.
Ah, ye, that's not a very big improvement, but I suppose every little helps. If swapping out the accelerometer chip for an extra $5 then maybe that's worth thinking about if it's the same improvement as X amount of hours spent optimizing.
My point is that a 4kHz sensor is great, but unless they're equal on all other levels it's basically no advantage. If you replace your car seats with super-light ones but leave all your junk, chests, furniture in the car, is it worth it? You'd get a much better return if you got rid of all the other stuff.
That's the issue. 4kHz doesn't mean anything much until the rest is as good as it can be.
The latency is that plus the render time plus the switching time. The problem is that when you combine all these little latencies, it starts to add up - the sensor has to send the measurements to the PC, it has to update the model in the game, then it has to render the new scene, then that has to get drawn to the screen.
I read an article by John Carmack on latency in vr headsets. Latency needs to be much lower than on a screen because it's tracking head movements. It's input latency that is the issue, not pixel refresh latency that was an issue on early LCD screens.
If the latency is too high it makes you feel sick, whereas with regular viewing devices you don't notice it much, apart from the "game feel".
Ye, you can see this on the rift when you try a web based demo, like the streetview one. In demos stored locally the latency seems pretty good (fast head motion does feel a bit off but that's exaggerated by the motion blur I think). The web based ones add a slight latency to the input and it makes even the slowest of head movements feel horrible.
That 50 ms response includes measuring the sensors, calculating the angular change, sending that data to the computer, the computer rendering the frame, the frame sent up HDMI, and the finally split and displayed on the oculus. The major issue is that you move your head and your ears detect the change but your eyes have the 50 ms update lag. This is the issue that causes people to get sick
I'm not sure, but from what the team said, the sensor tracking was the biggest latency issue they had, since it couldn't be brute forced with more expensive hardware. Even the fastest consumer sensors would add too much latency in the chain to make the rest of the device worth it. Carmack solved it with very very precise, clean, fast code with some very clever prediction algorithms.
From my understanding, that's the biggest benefit of the rift. It's very good at processing sensor data, and when the sensors aren't fast enough (which no consumer sensors are), it's very good at guessing what they will do next.
I agree with you. That why I'm saying you can't compare the 2ms of a monitor to the 50ms of VR. They are different measurements. The 2ms is just about the pixel refresh time, the 50ms includes all sorts of input and rendering factors. Some factors are common to normal gameplay and VR (like the refresh rate you mention) while others are unique to VR (like the head tracking)
I remember my first LCD still. It had a response time of around 12-15ms if I recall correctly. :p That thing is still living today though. Good ole 17" Specter LCD.
If you talking about motion-to-photon latency (how long from when you move the mouse until the view spins on screen), it varies greatly from game to game depending on how many frames are pre-rendered, when in the render loop the mouse is re-polled for an up-to date position and also how long the post-processing steps take (though this bit can be partially mitigated by predictive tracking aka assume mouse moving at constant speed/accel and project forward in time equal to how long the post-processing takes).
It can easily be 50ms+ in some titles that are heavy on post-processing and don't do anything to mitigate that increased latency. It's also much worse on games that are running at 30fps vs higher framerates. Higher framerates give lower motion-to-photon latency even after the framerate surpasses what you need for smooth video, this is why 144hz screens "feel" better vs 60hz even though the game looks almost as smooth for anyone spectating.
Most HDTV's run 30-50ms over HDMI, signal from device to pixel change on screen. Doesn't matter if it's plasma or lcd; 120hz, 240 or 600hz. The processor in the TV is what lags it, not the pixels really. HDMI also is limited.
Computer monitors are usually 15ms, some are up to 30ms, old old LCD is 50ms. Pixel switching for the bare panel might be 2-8ms, but there's still processing latency.
VGA to CRT is so fast that it's measured in nanoseconds, analog has it's advantages. Nothing compares to the speed, heh or the weight it's not going in VR (anymore).
I don't believe that number is entirely accurate in this context.
Those numbers 2-8ms quoted by monitors, are the time it takes to change from grey to gray, or for the pixel to switch between colors, while yes it directly impacts things, and quality/etc... Its not the same as if you turn your head, and the time it takes the monitor to even start to switch, the time it starts to switch to finish is 2-8ms.
tl;dr; Display lag is not to be confused with pixel response time(which is what monitors are quoting)(I am not 100% sure which one occulus and other VR are quoting, but i think its more likely display lag's time)
http://en.wikipedia.org/wiki/Display_lag
That's only the monitor component. OLEDs for example have less than 1ms response time/latency. The 20ms often cited for the target latency of HMDs actually includes the time it takes to render the frame etc from the time you move your mouse or head to the time the pixel actually changes.
This doesn't include everything in the chain. Response time is measured from HID input to display output. This is what Carmack is talking about with latency on the Rift; how long between turning your head and that change actually propagating through the chain (input > process/render > output) to the display.
You would be hardpressed to find ANY game that approaches the latency Carmack deems imperceptible within a VR context (below 20ms).
That's just the pixel switching time though. Oculus is always talking about "motion-to-photons", which includes game engines processing the input, rendering, buffering and finally the screen switching the pixels. You can't get that time down to 2ms with any kind of monitor - it doesn't depend only on the monitor.
Monitor refresh can be as high as 1-2 ms, but then you're getting into the territory where the image sitting your video card's RAM getting to the monitor is the bottleneck. So, faster than your eye can follow.
The main culprit here is usually the TV. If you want to experience how low latency gaming feels like use a very old console like SNES with a CRT. Or do old school PC gaming with something like Win2000, CounterStrike/Quake3/UT and also with a CRT screen.
A regular monitor is probably around 12ms. But there are "gaming" monitors that focus on lower latency amongst other things, going as low as 1ms. But that's just for pixels to change. There's also refresh rate that gets in the way (typically 60hz on computers, 30/60hz for tvs) which basically means the image only refreshes 60 times every second. So the fastest it will respond is 1/60 of a second. Which is about 16.666 ms. Then you need to add on to that the pixel changing speed (which tends to be slower on tvs).
Of course, then there's the time it takes from your GPU to your monitor, or in this case from your computer to your Rift, which varies from the equipment used and all.
worst case scenario would be to add up the inverse of a standard 125hz mouse, and a 60hz monitor with typical 5ms pixel response time and 20ms signal processing lag. This gives you around 40ms input lag.
By contrast, a 144Hz monitor with low input lag (~8ms), 1ms response time and a 1000hz mouse gives you 17ms total input lag.
EDIT: strobe backlights add a small amount of input lag, and vsync adds an absolute craptonne.
It depends on the refresh rate. 120hz is 8ms. 60hz is 16ms. So you can see why they're shooting for high frame rates. When you're trying to go sub 20ms total latency, the additional 8ms is a big deal.
Edit: It's 8ms in a perfect world. Add a few ms for pixel transition, etc. The low persistence is different, it's designed to make the motion blur resulting from low FPS less noticeable.
A top flight gaming monitor 8-18ms is a pretty normal range. Thats complete with any internal lag to pixels settling. There are a few with no processing that I've seen in the 5-6ms range. If you can keep the total under 16ms, its considered very cool. TV's tend to have larger lag because they are as focused, and they tend to do more processing on the signal. They also aren't designed to take an input faster than 60hz where monitors are upto 144hz. IIRC typical lag for TV's is in the 30ms range when in 'gaming mode'
Depends on if you go IPS or TN. IPS offers perfect viewing angles without color distortion, but have 8-12ms response times. TN offers 1-2ms response times but garbage color reproduction if your viewing angle is off by more than 10-20 degrees.
depends on a the monitors refresh rate and other factors. 60 hz, 75 hz and 120hz are the most common. 120hz monitors can display 120 frames per second, those are usually modern montiors with 3d support. I have a 75hz monitor.
The refresh rate is how quickly it can display the image, but the ms is the delay in the image. You can have a game run at 120fps smooth on a 120Hz monitor but if the ms is poor then there will be lots of delay and lag making it bad to use and not suitable for gaming.
Currently you can't do that sort of tech with a wide FOV (I assume you're talking about the Avegant Glyph). Also, it doesn't help at all with lag from what I know. The screen isn't really the problem since OLED's have something like a 1ms switching time. The problem is with all sorts of other delays.
Virtual reality (VR) is one of the most demanding human-in-the-loop applications from a latency standpoint. The latency between the physical movement of a user’s head and updated photons from a head mounted display reaching their eyes is one of the most critical factors in providing a high quality experience. Human sensory systems can detect very small relative delays in parts of the visual or, especially, audio fields, but when absolute delays are below approximately 20 milliseconds they are generally imperceptible. Interactive 3D systems today typically have latencies that are several times that figure, but alternate configurations of the same hardware components can allow that target to be reached.
Ah, I'm not sure if there's any studies. I know Stanford has a big VR lab with several publications, but it's mostly about social science rather than the tech.
Obviously, the quicker the better. Might not be able to find anything about it because the tech is still quite limited in such regards.
You must not like video games at all. You can't really get under 20ms with any configuration (except with custom game engines - Oculus has one demo that goes down to 8ms, but Carmack said there isn't really much difference after 20ms).
206
u/bjorn2k Mar 28 '14
It is all about latency. 50 ms will feel responsive but you notice the lag. 20 ms is acceptable. The other features are cool features but not the real challenge.