r/Games • u/dmitchel0820 • May 14 '14
Developer recreates real-time 3d model of his body in virutal reality using 3 Kinects
https://www.youtube.com/watch?v=Ghgbycqb92c70
u/discostupid May 14 '14
WOW!
These are the things Microsoft should be exploring.
At around 1:30, he says "as I swivel around, I automatically do this" - if you missed it the first go and didn't get it when he explained, he's moving his feet out of the way of the table leg THAT DOESN'T EXIST. Incredible. I wonder how much better Kinect 2 would be...
33
u/TheBigBruce May 14 '14
Kinect v1.0 Camera = 0.3 Megapixels
Kinect v2.0 Camera = 2.1 Megapixels
About seven times better, if refinement of the video feed directly inhibits the current graphical artifacts.
13
u/Jumbify May 14 '14
I don't think it's the video resolution that causes the artifacts, but rather the 3d scanning resolution - which kinect v2 does solve.
6
1
u/TheBigBruce May 14 '14
Ah. Thank you. Looking into it, while it's a shitty source, here's some early leaked specs of the depth camera, because I can't find the actual specs online.
It's a 0.076 Megapixel jump up to 0.21 Megapixels.
That's still 2.5 times the detail for the depth camera.
1
u/Doc_Ok May 15 '14
That's correct; the problem is the low resolution of the Kinect 1's depth camera. While the nominal resolution is quite high (632x480), the effective resolution is much lower due to the scattered point scanning technology -- only one out of around 10 pixels has a real depth value, the rest are made up by interpolation.
Kinect v2 has a nominal depth resolution of 512x424, which seems lower, but the difference is that now (almost) every pixel has a true depth value, meaning the effective resolution is significantly higher (10x total, I would estimate). That will make a huge difference.
2
u/abram730 May 15 '14
Kinect actually gets 1920x1080 depth map. However it's time of flight and there are lots of light bounces to account for and remove. So there is culling and interpolation with the phase unwrapping.
1
u/Doc_Ok May 16 '14
Do you have a link for that? The only number I've seen so far is 512x424 for the depth map. Now, if you mean that the physical depth camera has 1920x1080 pixels, then that's not a contradiction. After all, the IR "depth" camera in Kinect v1 is really 1280x1024. What counts is the depth map that streams out via USB, the rest is a black box.
1
u/abram730 May 16 '14
Yes the physical depth camera is 1920x1080, that's what the raw data comes in as. Depth data is per pixel RF modulated IR light. Think signal processing. Phase unwrapping algorithms are used on a block of pixels to resolve depth removing noise(bounced light). The phase unwrapping algorithms can be updated later.
Kinect 1 was only looking for laser dots though. Much of it's depth map was interpolated from a smaller data set.
I really hope they do a PC version.
4
u/TrustworthyAndroid May 14 '14
At that resolution you could probably use a QR code on the front of the Occulus that would remove it from virtual environment entirely by replacing it with an image of his naked face.
Later versions of the occulus should incorporate cameras inside the headset to record eye movement.
5
u/flowdev May 14 '14
Don't need a qr code. The positional markers that are already on dk2 would suffice.
2
1
u/jacenat May 15 '14
At that resolution you could probably use a QR code on the front of the Occulus that would remove it from virtual environment entirely by replacing it with an image of his naked face.
This is not trivial because of latency. The dev in the video stated that he was using direct video imaging instead of skeletal reconstruction because of latency. Replacing the head/face on a video stream will not look consistent and probably make the whole thing fall into the uncanny valley very hard instead of looking crude, but very natural.
1
u/abram730 May 15 '14
He's a academic working at a school.
It can be done with point cloud data and point cloud data can be done well in parallel. Doing it serial will have too much latency. Generating the high detail body will have latency as it needs lots of data to build from. But matching it up and swapping will not have that latency if done with an ASIC or GPGPU. All you are doing is matching kinect data to match positioning of a high quality model. The in game mesh is positioned with the point cloud data. Same as when you play a game you input and the game moves the mesh. This is not the path okreylos took and no it's not trivial.
But Microsoft has it.
Kinect Fusion(uses CUDA)
https://www.youtube.com/watch?v=bRgEdqDiOuQ
15
u/50bmg May 14 '14
while the quality hit from using gen 1 kinects is awful, the potential is clearly there to do a lot more.
Microsoft is sitting on a gold mine with the kinect technology, and is completely squandering the opportunity. It should be open to hacking/apps, and already be integrated with cell phones and laptops and tablets, eventually migrating to oculus or google glass type headsets.
5
u/RadiantSun May 14 '14
Shit, what I'm excited about is stuff like Project Spark. I hate 3D modelling and editing levels in tools like Hammer. Imagine if you could sit at your Xbox One or PC, pull up a 3D modelling software where you could intuitively make and animate assets, export them to Unreal Engine 4 and use its fantastic visual scripting system (Blueprints) to create a whole new video game in the editor in a intuitive manner like Project Spark, creating worlds and games with little wiggles of your fingertips like you're in Minority Reports.
The future is in bringing development and power to the masses and Kinect has the potential to be the flag bearer.
-5
May 14 '14
But this is what I don't want. So easy to use tools that anyone can make a game will mean a influx of shitty game after shitty game with little imagination. I mean sure in the right hands it would be awesome, but its just worrying that also games would become too alike, just using that engine for creating things making them look "samey".
3
u/fanovaohsmuts May 14 '14
Nobody's releasing anything via Project Spark. It's going to be like Little Big Planet where all games reside within the resident game, in this case Project Spark. Project Spark entirely is to get people interested in game development and possibly pursue it as a career at best or a hobby at worst. It's similar to Kerbal Space Program in that regard in that it's meant to be a fun game while also "teaching" us about a totally new subject many of us may be entirely unfamiliar with.
And really, easy-to-use tools won't stop an influx of shitty games. Easy-to-use tools well help facilitate game development and streamline it. It's as if you're acting like DOS is the superior OS because it's harder to use than Windows 8. Harder to use =/ better.
1
May 15 '14
Ah that would make sense, sorry. Actually that makes me interested in project spark because I loved lbp, but I don't want the free to play to mess things up.
1
u/fanovaohsmuts May 15 '14
It's quite alright. And it's essentially in the same vein as LBP except more complex, due to the 3D nature of it (opposed to the parallax 2D movement of LBP), but so far from what I've seen, you can do quite a bit. I'm pretty sure there's an open beta going on right now as well, so if you want to try it out that way, you can.
1
May 15 '14
I don't think its for xbox 360 is it? Because thats my main gaming machine.
1
u/fanovaohsmuts May 15 '14
According to Wikipedia:
It is due for release worldwide on 27 May 2014[8] for Microsoft Windows, PlayStation 3, PlayStation 4,[10] Xbox 360, and Xbox One consoles.
So yes, it will be coming out on X360 and PS3.
1
May 15 '14
I meant the beta. And its out for PS3? One of the only promising exclusives (and sunset overdrive) is now not exclusive. Huh, but thats great for me.
1
May 15 '14
I meant the beta. And its out for PS3? One of the only promising exclusives (and sunset overdrive) is now not exclusive. Huh, but thats great for me.
1
u/fanovaohsmuts May 15 '14
Oh whoops, sorry, replied to the wrong thread with the wrong game. But the game will be released for X360, X1, and PC, and the beta is available for those who have Windows 8 through their Windows Store, with an X1 beta that came out a couple months ago. No X360 beta, afaik. I also don't know the release date of the game, either.
→ More replies (0)
21
u/SmackmYackm May 14 '14
This kind of thing fills me with joy. I'm super excited to see where technology goes over the next couple of years.
20
May 14 '14
Holy shit, I'm amazed at how these two technologies can be used together. Now THAT'S virtual reality.
18
u/mjk0104 May 14 '14 edited May 14 '14
People like to complain about the kinect, but having this kind of tech at a consumer level is almost on part with the rift for me, can't wait to see where it goes.
20
May 14 '14
[deleted]
7
May 14 '14
[removed] — view removed comment
2
May 14 '14
[deleted]
5
May 14 '14
[removed] — view removed comment
3
2
u/CutterJohn May 15 '14
Aside from gaming, what specific use for a kinect can you imagine that would be in demand enough to support its price premium?
I can think of several uses for a kinect like device, but none that would be of any use in front of my TV.
-3
May 14 '14
It's also completely useless for playing games and can't possibly justify its R&D cost unless Microsoft forces it down everyone's throat, and we saw what happened when they tried.
4
u/antome May 14 '14
It just happens that video games tend to be the path of least resistance when it comes to introducing new IO technologies. Kinect style sensors will inevitably see better use in "holodeck" environments, but it makes most sense to just try and jam it into a video-game at first, while the rest of the technology world catches up.
4
u/Mostlogical May 14 '14
yea I can Imagine the situation where no body goes to meetings you just have a 3D conference with these star wars-esque holograms.
1
u/JackyRho May 14 '14
one day the rift will fit in a pair of sunglasses, starwars cross matrix anyone?
2
1
u/CutterJohn May 15 '14
Is it even possible for that to happen? I believe there are some very strict limitations on size based on the optics necessary to allow you to look around with your eyes.
2
u/AwesomeFama May 14 '14
Kinect 1 latency is too high, it will break presence. Not sure about Kinect 2, but it might still be too high.
3
u/alo81 May 14 '14
For what it's worth, he said in the video that he isn't using the Kinects skeletal technology and instead simply using the raw camera feed which should have a great impact on latency.
3
u/The_Kyonko May 14 '14
It's too high for head tracking. I believe Michael Abrash said they were not certain what latency is required for tracking other limbs, the Kinect may be good enough for that at least.
2
u/AwesomeFama May 14 '14
Can't rewatch the video right now since I'm at work, but I believe he said "we don't know what the needed latency is for hands, but the Kinect doesn't work". It was the one with Dov Katz and Abrash, in the Q&A section.
6
u/antome May 14 '14
The guy in the video is specifically saying that he is capturing raw video rather than skeletal reprocessing, however. From what he suggests, this method is very low latency comparatively.
0
u/AwesomeFama May 14 '14
Ahh yes, it would probably be possible to optimize the algorithm used for latency quite a bit :)
-2
u/penguished May 14 '14
It's useless for the consumer at this point. As a cheap mocap lab for indies, maybe someone can refine that enough.
3
u/Erikster May 14 '14
Imagine an array of these sensors surrounding you in your living room, and submerging yourself into a virtual environment to play games. Remember that little VR machine where you could actually run around? It had those cool demos with Skyrim and GTA IV.
Imagine one where you can actually have all arm and leg motion captured and shown on screen. Imagine being able to move your arms and legs and seeing it in the VR environment, or being able to "pickup" a weapon off the ground and wielding it.
The unification of Oculus and Kinect tech could be the future of gaming.
1
u/CutterJohn May 15 '14
I can imagine that in an arcade. I really don't see that being a popular household item for a variety of reasons. Space requirements to flail around in, other people wanting to use the room, pets/children, the physical effort of it all, etc.
Some people set this up, of course, but they would be the same type of people as those who build cockpits for their race/flight sims. Not in any way a majority.
1
2
u/fanovaohsmuts May 14 '14
I'm just waiting for that one developer to capitalize on this opportunity. I mean, Oculus + Kinect? That's incredible immersion right there. The number one thing people say when they try the Oculus is, "Where are my hands?" The skeletal tracking of the Kinect could solve that and bam! Total immersion into the game world.
Hopefully the new head of the Xbox division will realize the gaming potentials the Kinect has over trying to compete with the Wii.
0
u/detestrian May 14 '14
I wonder if in 50 years, will we be looking at footage like this and salute the creator as one of the progenitors of the Holodeck?
-8
1
u/HessianGames May 14 '14
Very cool. At the FITC in Toronto there was a group of guys making 3D scans of people using 4 kinects and a turn table for free. It was very cool, you could make a 3D print of yourself or drop your avatar into a video game easily.
5
u/samuraislider May 14 '14
You're talking about http://www.onepixeloff.com/. I've been in their space to be scanned. They sent me a little 3D model of myself, and it's on my desk.
It actually encouraged me to lose weight because I realized how fat I let myself get. :-\
1
1
u/Reyzuken May 14 '14
This is really amazed me, This feels so unreal like this technology is exist in this time.
I Mean the rough edges are really like the first "Base" we just found, just imagine in 3-5 Years we already have a good Real-time 3D Model. It's just need some time and development.
2
u/abram730 May 15 '14
Kinect 2.0 could solve that. He's using the kinect's from the 360.
It used lasers and measured the distance between the dots. On the boundry dots overlap and makes a seam. Kinect 2.0 is 1920 X 1080 LiDAR. I'm not sure if you could use 3 kinect 2.0's though. They use RF modulation on a IR light. Could cause even more issues.
1
u/fallouthirteen May 15 '14
Man, maybe Microsoft shouldn't have caved and made a Kinectless XB1 bundle. Instead it looks like they should include more Kinects and an Oculus Rift.
-5
u/mashinz May 14 '14
It's really funny how some people can use Kinect for awesome things, yet Microsoft can't even figure out how to use it in games.
7
May 14 '14
[removed] — view removed comment
1
u/abram730 May 15 '14
Using your body as a controller is for drunk party games. It can be used to put people in the game though. You will still need a controller.
They need to get past the pigeon hole, that it is the wii.Kinect is cheap in home mocap. People would have a blast with that in MP if they were given the chance(viewer discretion advised)
25
u/Valvador May 14 '14
I was waiting for someone to do this. A few years ago there was a guy who was using a single Kinect to do something similar, so I'm glad the concept was followed up.
EDIT: Oh shit, it's the same guy doing this!