Reddit post I wrote in a very tired fugue state after shipping SMNC. I've been inspired to think about this stuff by the Google and Valve projects.
http://www.reddit.com/r/programming/comments/sjgm5/michael_abrash_valve_how_i_got_here_what_its_like/c4eud6a
Ok let me take a shot at it.
Warning: extremely long week, very tired, just launched our game this week. I may not be writing coherently at the moment.
First off there are definitely technical problems as you've pointed out.
I would say the most difficult thing to nail is going to be the display. As you've pointed out eye strain can be a problem for example. You also need to be compatible with glasses and probably contacts. There are interesting pieces of tech that can do a good job of this. Currently these are mostly military in nature but there is a company close to Valve that has been working on this for years. Check out Microvision for example. They make laser based displays including pico projectors that are quite sweet. This company has been languishing lately and Valve could buy them for <$100M. In the medium term this is a solved problem. One of the really nice things about these laser based displays is the low latency making high refresh rates possible which is important for my next point
The next technical issue is tracking your head movement quickly. If you want to render anything "solid" into the world in 3D you would need to be able to keep up with head movement very quickly. Otherwise you tend to make people sick. This is mostly what killed this tech during the 90's. The key to solving this is low latency algorithms paying attention to the environment using the cameras as well as the tilt sensor tech that things like an iphone uses. We need more work on the correct algorithms to really make the display solid as you look around and have things anchored correctly. If we can get the latency down to a couple of ms well be fine (say 250hz or better) based on my research.
Now once we have glasses that have a low latency "solid" display that can track the world properly we can start doing really interesting things. With multiple cameras and other tricks we can get a depth buffer for the world and composite our own images in 3D.
The Google glass demo was dumb as you said. They didn't have any imagination and are basically showing off a phone in the corner of your vision. Bah, lame.
So here are a few ideas I think would be cool. Early on it's going to be difficult to "read" the world and figure out what's going on so you can do intelligent stuff. When I think of something with a regular pattern the first thing that springs to mind is instruments.
Guitars have a very regular fret board that we should be able to track. Imaging wearing the glasses and having the guitar teach you chords by overlaying onto your vision. Imagine a game like Rocksmith that you play with the glasses on that can teach you songs, play along etc. You could even do a guitar tutor that takes you from beginner all the way to advanced. Want to always be reminded of which notes are where? Want a built in tuner that hangs in your vision? Want to see a display breaking down the details of your sound in the corner of the room? Want to have notes hover in the air while playing? Want virtual amp controls? Want to be standing on a stage in front of a huge audience? Want to share the stage with your favorite band? Tell me that's doable with a phone.
Now think about pianos and other instruments. Anything with regular patterns should be fairly easy to get right. Hell the piano doesn't move it would be even easier than the guitar. Picture all the notes labeled. Picture it showing you how to play songs etc. You get the picture.
Ok, on to other things. Gaming is really obvious. RTS games that take place in shared virtual table top space. Everyone wears glasses and sees the same "board". You could have spectators using a web cam or share your feed with the net. You could have players with their own boards in other cities. Just imagine an overlay for something like Settlers that keeps track of what goings on, has animations etc. Family game night from across the continent. Board games alone translated and made into interactive experiences would be huge.
I could go on and on about the game possibilities. Real life laser tag with your friends. It could have explosions, damage, health bars above people whatever you want. Basically bring games into the real world. Location based games could do crazy stuff with this tech. For example you could play a country wide deathmatch style game that was opt-in. Whenever you saw an enemy player in public you engage. Virtual gangs coming to a hood near you.
There is also the obvious avatar spec type stuff. Everyone agrees on an avatar spec system and if you so choose you can squawk what you are supposed to look like. Permanent cos play without a need to wear real clothes. I would expect multiple groups would co-exist so you might look different depending on which group someone is looking at. Or they could override it like you can with a ring tone today.
How about commercial applications? I can think of a ridiculous amount of applications in that space. The military uses this stuff for a reason. Plenty of work could benefit from a heads up display including IMHO coding. We can always use more monitor space.
All of the obvious world overlay stuff Google showed. Basically knowing anything Google knows about an object just by paying attention to it. Can you imagine getting used to that level of information flow?
Watch movies anywhere on a "big" screen. In 3D.
How about a full streamed recording of your life bonded and encrypted in the cloud? Want to know what you did on a particular day? control-shift-n to blank it out for a bit ;)
I also think that the form factor of glasses makes more sense than a little block box with a tiny screen. My prediction is that within a decade this technology replaces current cell phones in a ubiquitous way. It's the next generation of smart phone and once it's worked out you'll want one the same as everyone else. The future is going to be awesome. Now I just want a flying car.
http://www.reddit.com/r/programming/comments/sjgm5/michael_abrash_valve_how_i_got_here_what_its_like/c4eud6a
Ok let me take a shot at it.
Warning: extremely long week, very tired, just launched our game this week. I may not be writing coherently at the moment.
First off there are definitely technical problems as you've pointed out.
I would say the most difficult thing to nail is going to be the display. As you've pointed out eye strain can be a problem for example. You also need to be compatible with glasses and probably contacts. There are interesting pieces of tech that can do a good job of this. Currently these are mostly military in nature but there is a company close to Valve that has been working on this for years. Check out Microvision for example. They make laser based displays including pico projectors that are quite sweet. This company has been languishing lately and Valve could buy them for <$100M. In the medium term this is a solved problem. One of the really nice things about these laser based displays is the low latency making high refresh rates possible which is important for my next point
The next technical issue is tracking your head movement quickly. If you want to render anything "solid" into the world in 3D you would need to be able to keep up with head movement very quickly. Otherwise you tend to make people sick. This is mostly what killed this tech during the 90's. The key to solving this is low latency algorithms paying attention to the environment using the cameras as well as the tilt sensor tech that things like an iphone uses. We need more work on the correct algorithms to really make the display solid as you look around and have things anchored correctly. If we can get the latency down to a couple of ms well be fine (say 250hz or better) based on my research.
Now once we have glasses that have a low latency "solid" display that can track the world properly we can start doing really interesting things. With multiple cameras and other tricks we can get a depth buffer for the world and composite our own images in 3D.
The Google glass demo was dumb as you said. They didn't have any imagination and are basically showing off a phone in the corner of your vision. Bah, lame.
So here are a few ideas I think would be cool. Early on it's going to be difficult to "read" the world and figure out what's going on so you can do intelligent stuff. When I think of something with a regular pattern the first thing that springs to mind is instruments.
Guitars have a very regular fret board that we should be able to track. Imaging wearing the glasses and having the guitar teach you chords by overlaying onto your vision. Imagine a game like Rocksmith that you play with the glasses on that can teach you songs, play along etc. You could even do a guitar tutor that takes you from beginner all the way to advanced. Want to always be reminded of which notes are where? Want a built in tuner that hangs in your vision? Want to see a display breaking down the details of your sound in the corner of the room? Want to have notes hover in the air while playing? Want virtual amp controls? Want to be standing on a stage in front of a huge audience? Want to share the stage with your favorite band? Tell me that's doable with a phone.
Now think about pianos and other instruments. Anything with regular patterns should be fairly easy to get right. Hell the piano doesn't move it would be even easier than the guitar. Picture all the notes labeled. Picture it showing you how to play songs etc. You get the picture.
Ok, on to other things. Gaming is really obvious. RTS games that take place in shared virtual table top space. Everyone wears glasses and sees the same "board". You could have spectators using a web cam or share your feed with the net. You could have players with their own boards in other cities. Just imagine an overlay for something like Settlers that keeps track of what goings on, has animations etc. Family game night from across the continent. Board games alone translated and made into interactive experiences would be huge.
I could go on and on about the game possibilities. Real life laser tag with your friends. It could have explosions, damage, health bars above people whatever you want. Basically bring games into the real world. Location based games could do crazy stuff with this tech. For example you could play a country wide deathmatch style game that was opt-in. Whenever you saw an enemy player in public you engage. Virtual gangs coming to a hood near you.
There is also the obvious avatar spec type stuff. Everyone agrees on an avatar spec system and if you so choose you can squawk what you are supposed to look like. Permanent cos play without a need to wear real clothes. I would expect multiple groups would co-exist so you might look different depending on which group someone is looking at. Or they could override it like you can with a ring tone today.
How about commercial applications? I can think of a ridiculous amount of applications in that space. The military uses this stuff for a reason. Plenty of work could benefit from a heads up display including IMHO coding. We can always use more monitor space.
All of the obvious world overlay stuff Google showed. Basically knowing anything Google knows about an object just by paying attention to it. Can you imagine getting used to that level of information flow?
Watch movies anywhere on a "big" screen. In 3D.
How about a full streamed recording of your life bonded and encrypted in the cloud? Want to know what you did on a particular day? control-shift-n to blank it out for a bit ;)
I also think that the form factor of glasses makes more sense than a little block box with a tiny screen. My prediction is that within a decade this technology replaces current cell phones in a ubiquitous way. It's the next generation of smart phone and once it's worked out you'll want one the same as everyone else. The future is going to be awesome. Now I just want a flying car.
No comments:
Post a Comment