Slashdot Log In
Head Tracking w/ the Wiimote
Posted by
CmdrTaco
on Sun Dec 23, 2007 10:04 AM
from the this-is-just-too-cool dept.
from the this-is-just-too-cool dept.
mrneutron2003 writes "This guy just doesn't know when to stop. Johnny Chung Lee graces us with yet another one of his inventive Wiimote projects. This time it involves using the Wiimote and a pair of inexpensive LED safety goggles (with the standard LED's replaced with InfraRed ones) to allow positional head tracking , achieving an effect similar to what is experienced with three dimensional displays and CAVE systems. The video dramatically illustrates the effect. Game developers take note. This simple little variation on infrared tracking could allow for some seriously immersive gameplay in the future." This guy deserves a medal.
Related Stories
[+]
Wiimote as Multi-Touch Display Controller 107 comments
Tmack writes "While hard-hacks with the Wiimote are somewhat old news, this particular implementation is quite interesting. Using the infrared camera on the Wiimote, pens with LEDs instead of ink, and an LCD projector, Johnny Chung Lee of Carnegie Mellon University has created software to use them as a (relatively) cheap multi-touch display. Any surface onto which you can project becomes an interactive multi-touch display, as demonstrated in the video at the link. He has the software available for download, along with some other neat projects.
Lee has also documented another impressive Wiimote hack.
[+]
Technology: Wiimote Turns TV into Touchless MS Surface 104 comments
RemyBR writes "User interface project allows you to control objects on a display using gestures, working like Microsoft's Surface but without touching the screen at all. Inspired by Johnny Chung Lee's work, the system requires you to wear Minority Report-style gloves equipped with infrared emitters on your fingertips. A Wiimote on top of the display keeps track of these IR LEDs, while the software can read the motion down to two-finger pinching gestures for image zooming."
[+]
Modern Methods For Sharing Innovation 91 comments
The New York Times is running a story about Johnny Chung Lee, a hardware hacker made famous for his projects which modified the Nintendo Wiimote to do things like positional head tracking and multi-touch display control. The article focuses on the suggestion that Lee's use of YouTube to demonstrate his innovations has done a better job of communicating his ideas than more traditional methods could. Quoting:
"He might have published a paper that only a few dozen specialists would have read. A talk at a conference would have brought a slightly larger audience. In either case, it would have taken months for his ideas to reach others. Small wonder, then, that he maintains that posting to YouTube has been an essential part of his success as an inventor. 'Sharing an idea the right way is just as important as doing the work itself,' he says. 'If you create something but nobody knows, it's as if it never happened.'"
[+]
Games: Why Natal Is a Big Deal 303 comments
Kikizo has an editorial piece evaluating the Xbox 360's upcoming motion-control scheme, Project Natal, and discussing why it's a bigger step forward for interactive gaming than many people think. Quoting: "[Natal] accurately perceives players in 3D space, simultaneously tracking over 48 joints on your body, enabling it to accurately redraw your skeleton in real time as you move about. On a separate 'debug screen' in the closed-doors session, we could witness for ourselves the 'mind's eye' of Natal, visually showing how it completely understands where we are, how we're moving, where we are in 3D space, how far in front of my face my hand is, whatever. It can supposedly even track individual hand and finger movement when it switches into this more finely-tuned mode. ... There is a surprising feeling of tactility and iPhone-like fluidity and precision to the way Natal works." Another interesting bit of news about Natal is that Wii-hacker Johnny Chung Lee is part of the development team. We've discussed some of his creations in the past.
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
Nintendo! Hire Johnny Lee! (Score:5, Insightful)
Re:Nintendo! Hire Johnny Lee! (Score:5, Funny)
Parent
Re: (Score:3, Informative)
Re: (Score:2)
Re: (Score:3, Interesting)
Keyboards on consoles ain't news. That's about as stale as a Sega Mega Drive. Not to mention that people would most likely rather use a "normal" computer for browsing the web.
This is a new feature. Brand new. And doesn't need any additional gadgets or any new developments. At the very least, Nintendo should have had something like that demo in its standard applications to show it off and leave the customer dreaming of games that make use of it.
Re:Nintendo! Hire Johnny Lee! (Score:5, Insightful)
The PC Gaming landscape is littered with failed head-tracking systems. The reviews inevitably say something like "this thing is awesome, but fatiguing."
There are eye-tracking systems that are not nearly as fatiguing, but if you've seen one, you'll understand why they haven't taken off in popularity.
Parent
Re: (Score:3, Insightful)
If you talked about putting a accelerometer in to a controller before the Wii, you'd be laughed at.
Re: (Score:3, Insightful)
Re:Nintendo! Hire Johnny Lee! (Score:5, Insightful)
If you're perfectly perpendicular to your monitor, there is limited arc of motion that your head can make before the monitor is out of your direct line of sight and into your peripheral vision. This artificially limits what you can do in a game and is why head tracking systems have not replaced traditional controls for looking along the X & Y axis.
I'm not saying there is no role for this in gaming, I think it would be great if Nintendo could make it cheaply for the Wii and developers created games that could use it effectively... but that has been tried before in PCs... without much success.
Parent
it's all research, man (Score:2)
I guess he'll have to settle for a PhD from Carnegie Mellon instead.
Re:it's all research, man (Score:4, Insightful)
Parent
Re: (Score:3, Insightful)
How about people just send him some money so he'll keep doing what he's doing and make it free?
Re:Nintendo! Hire Johnny Lee! (Score:4, Insightful)
Only problem now is that i can't see the TV
And besides, it's not what direction you're looking, it's what direction you're looking from. Move your whole body to the right while continuing to look at the TV and the display on the TV changes perspective. Not to mention depth of field, and distance from the TV. Did you even watch the video?
Why am I even responding to an AC comment?
Parent
video down (Score:5, Informative)
DUDE (Score:2)
I can only imagine what something that that, coupled with a graphics engine like Assassin's Creed has would do for immersive gaming.
Re: (Score:2)
Polarized filters and LCD shutters aren't anything new, and I'm sure they've been combined with head tracking before... But I don't think it's ever been quite as accessible as it is now. Stereoscopy + head tracking + a pointing device like the Wii remote could make for one hell of game.
Polarized light not compatible with LCD (Score:5, Informative)
They only work with DLP projectors (uses little mirrors), CRTs, plasma, and upcoming display technologies like Field Effect Displays and LED displays. Obviously there are a lot of display technologies that do work there, but LCD is a very popular technology for widescreen TV and of course, for PC monitors.
Either way you do it, you also have to double the grunt of your rendering system (or half your graphical complexity), and you need specific software support to get it right (you can go a long way with a driver that knows it's rendering for stereoscopy and just produces the correct eye POVs, but the glitches you get in the foreground and HUD are only tolerated by enthusiasts.). With shuttering you need glasses. With cross polarization you need to double the number of display elements (by having two displays or a special display with double the horizontal resolutions). Used in POV applications, all of these technologies are a one-user gig.
Stereo "Wii-D" will probably never happen ; half the audience have an incompatible display device, the system does not have an enormous excess of GPU grunt. Stereo3D would only be common with one of the following display devices...
* Personal head-mounted 3D display (probably VRD goggles)
* Large area wide aspect flatpanel displays with inherent stereo 3D support built in at the factory (which means basically doubling the vertical rez and making a special polarized filter for the screen).
The parallax effect that Johnny Lee demonstrates conveniently exploits the tendency of the human brain to "fill in the gaps" ; I'd be intrigued to see how convincing it really is.
As another poster points out, head tracking really isn't very well received for the PC, because the PC is an inherently static device. You can move your head, but your hands have to remain fixated on the keyboard / mouse. The Wii has an advantage here because the input device moves around with you. Several times during Zelda I got up from my chair and started moving almost involuntarily, my whole body was immersed in the game. I would never have tried that on the PC ; when I feel the urge there it probably just contributes to my neck tension.
If the static, 3rd person POV of Zelda can make this gamer rise up and move, a game armed with a head tracking linked POV would be compulsively immersive, even without stereoscopic 3D.
Parent
Muppets? (Score:2, Insightful)
Also, the head tracking is awesome.
Re: (Score:2, Offtopic)
As for the projects he's doing, you're right, it's awesome. Even watching the youtube video of the headtracking gives you the feeling that you were watching a 3D effect, I wonder how i
Aiming a gun by looking at your target (Score:2)
The Wiimote is truly the ultimate hackable peripheral...
Re:Aiming a gun by looking at your target (Score:5, Interesting)
Parent
Wiimote with ability to track more points? (Score:2)
Re: (Score:2, Insightful)
Worth mentioning.. (Score:5, Informative)
The TrackIR solution linked above costs around as much as a Wii itself.
Re: (Score:2)
This little software feeds the TrackIR interface from a cheap webcam. Since it doesn't use a Wii it won't make you more attractive to the gender that you prefer being attractive to, but then you get full six degrees of freedom, which a Wii sensor bar can't do (it has only two points to track). Only trouble is the weak framerate most webcams have, and horrible webcam drivers that can suck away a considerable amount of CPU time.
Why isn't this guy working for nintendo? (Score:2)
Of course, having a 42" plasma or larger is going to be the optimal thing here - and since the wii is priced to sell, then a lot of people are going to be using them on their old 1980's 27inch CRT's.
Re: (Score:2)
Insensitive clod, I bought mine in the 90's!
Fantastic (Score:5, Interesting)
Re: (Score:2, Informative)
Better than a medal (Score:4, Insightful)
Nintendo, are you listening?
Editors need to edit. (Score:5, Informative)
You spelled 'too' wrong.
Re: (Score:2)
Just be thankful he didn't spell it "two"
Remind me again... (Score:3, Insightful)
Combine this with the weight-shifting capability of the Fit, and you've got an immersive gaming experience that's second only to the holodeck.
So. Freaking. Cool.
Re: (Score:2)
Just because somebody does cool stuff with a Wiimote on a *PC* doesn't mean you will *ever* see something like this on the Wii. Remember this is homebrew stuff, not 'cool new games' that Nintendo will be releasing next year. The issue isn't that you couldn't have fun with the Wii, but that Nintendo really doesn't release much interesting stuff.
Remember the PowerGlove? (Score:2)
It's been done. Remember the Nintendo PowerGlove [wikipedia.org]?
Incidentally, if you've never tried gloves-and-goggles VR, it's cool for about ten minutes. Trying to do things by making gestures in the air is a huge pain. Without tactile feedback, it's tiring and inaccurate. I tried most of the VR systems in the first round, including Jaron Lainer's original system. No good.
It might not suck if the system had an end to end lag of under 10ms. "Turn head, wait for view to catch up" systems drive the user nuts. T
Re: (Score:2)
Apple confronted this with the iPhone touch screen, IIRC, and solved it by having the phone vibrate ever so briefly when a touch was registered. This gave the sensory impression of a button clicking without actually doing so.
I don't see why a developer couldn't do something like this with VR gloves, then. Using JLC's approach, you'd have "Minority Report"-style gloves that had infrared ref
Johnny Lee Rocks! (Score:5, Interesting)
This will likely spur an avalanche of Wii hacks, and could easily cause wiimote sales to go thru the roof..
I'm totally enjoying the adventure Johnny!
Combinations? (Score:4, Interesting)
You don't need a Wii or the remote. (Score:3, Insightful)
Potentially you could just use a webcam with an IR filter in front of it instead of a Wii remote.
Note: 1) there is usually a filter to filter out IR inside most webcams, so that would have to be removed. 2) IR emitter tracking would have to be done on the PC instead of inside the Wii remote.
Re:You don't need a Wii or the remote. (Score:4, Informative)
Parent
HOLY SHIT (Score:4, Insightful)
Why just limit this to games? (Score:3, Informative)
If you had several monitors, this could be used to make them feel as if they were an actual "pilots seat" of a vehicle giving perfect perspective to the "pilot" because they know where the head is oriented and each monitor could produce the proper peripheral and image views for the "pilot"
It would take a little tricky camera work for the robotic vehicle, but I am sure gratuitous funding could solve those problems.
Split Screen? (Score:3, Interesting)
Re: (Score:2)
Re:processing power (Score:5, Interesting)
Parent
Re:processing power (Score:5, Interesting)
Not to trivialise what Johnny is doing there is basically measuring the position of the wiimote in relation to the sensor bar - something it already does. The code to do this shouldn't be that difficult. The true genius was in him realising that you could do this easiest by reversing the moving component and the stationary component.
Apart from some smoothing algorithms, this is no more complex than reading the wiimote's pointer position and mapping that to a camera viewpoint.
Parent
Re:processing power (Score:5, Insightful)
It's so simple that you can do something with it, without having to wait for IBM, or Nintendo or any other big-$$$ company to bring out the relevant hardware in maybe 5 years.
Parent
Re: (Score:3, Interesting)
Re:processing power (Score:4, Insightful)
Parent
Re: (Score:3, Informative)