Slashdot Log In
Head Tracking w/ the Wiimote
Posted by
CmdrTaco
on Sun Dec 23, 2007 11:04 AM
from the this-is-just-too-cool dept.
from the this-is-just-too-cool dept.
mrneutron2003 writes "This guy just doesn't know when to stop. Johnny Chung Lee graces us with yet another one of his inventive Wiimote projects. This time it involves using the Wiimote and a pair of inexpensive LED safety goggles (with the standard LED's replaced with InfraRed ones) to allow positional head tracking , achieving an effect similar to what is experienced with three dimensional displays and CAVE systems. The video dramatically illustrates the effect. Game developers take note. This simple little variation on infrared tracking could allow for some seriously immersive gameplay in the future." This guy deserves a medal.
Related Stories
[+]
Games: Wiimote as Multi-Touch Display Controller 107 comments
Tmack writes "While hard-hacks with the Wiimote are somewhat old news, this particular implementation is quite interesting. Using the infrared camera on the Wiimote, pens with LEDs instead of ink, and an LCD projector, Johnny Chung Lee of Carnegie Mellon University has created software to use them as a (relatively) cheap multi-touch display. Any surface onto which you can project becomes an interactive multi-touch display, as demonstrated in the video at the link. He has the software available for download, along with some other neat projects.
Lee has also documented another impressive Wiimote hack.
[+]
Technology: Wiimote Turns TV into Touchless MS Surface 104 comments
RemyBR writes "User interface project allows you to control objects on a display using gestures, working like Microsoft's Surface but without touching the screen at all. Inspired by Johnny Chung Lee's work, the system requires you to wear Minority Report-style gloves equipped with infrared emitters on your fingertips. A Wiimote on top of the display keeps track of these IR LEDs, while the software can read the motion down to two-finger pinching gestures for image zooming."
[+]
Technology: Modern Methods For Sharing Innovation 91 comments
The New York Times is running a story about Johnny Chung Lee, a hardware hacker made famous for his projects which modified the Nintendo Wiimote to do things like positional head tracking and multi-touch display control. The article focuses on the suggestion that Lee's use of YouTube to demonstrate his innovations has done a better job of communicating his ideas than more traditional methods could. Quoting:
"He might have published a paper that only a few dozen specialists would have read. A talk at a conference would have brought a slightly larger audience. In either case, it would have taken months for his ideas to reach others. Small wonder, then, that he maintains that posting to YouTube has been an essential part of his success as an inventor. 'Sharing an idea the right way is just as important as doing the work itself,' he says. 'If you create something but nobody knows, it's as if it never happened.'"
[+]
Games: Why Natal Is a Big Deal 303 comments
Kikizo has an editorial piece evaluating the Xbox 360's upcoming motion-control scheme, Project Natal, and discussing why it's a bigger step forward for interactive gaming than many people think. Quoting: "[Natal] accurately perceives players in 3D space, simultaneously tracking over 48 joints on your body, enabling it to accurately redraw your skeleton in real time as you move about. On a separate 'debug screen' in the closed-doors session, we could witness for ourselves the 'mind's eye' of Natal, visually showing how it completely understands where we are, how we're moving, where we are in 3D space, how far in front of my face my hand is, whatever. It can supposedly even track individual hand and finger movement when it switches into this more finely-tuned mode. ... There is a surprising feeling of tactility and iPhone-like fluidity and precision to the way Natal works." Another interesting bit of news about Natal is that Wii-hacker Johnny Chung Lee is part of the development team. We've discussed some of his creations in the past.
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
Nintendo! Hire Johnny Lee! (Score:5, Insightful)
Re:Nintendo! Hire Johnny Lee! (Score:5, Funny)
Parent
Re:Nintendo! Hire Johnny Lee! (Score:5, Insightful)
The PC Gaming landscape is littered with failed head-tracking systems. The reviews inevitably say something like "this thing is awesome, but fatiguing."
There are eye-tracking systems that are not nearly as fatiguing, but if you've seen one, you'll understand why they haven't taken off in popularity.
Parent
Re: (Score:3, Insightful)
If you talked about putting a accelerometer in to a controller before the Wii, you'd be laughed at.
Re:Nintendo! Hire Johnny Lee! (Score:5, Insightful)
If you're perfectly perpendicular to your monitor, there is limited arc of motion that your head can make before the monitor is out of your direct line of sight and into your peripheral vision. This artificially limits what you can do in a game and is why head tracking systems have not replaced traditional controls for looking along the X & Y axis.
I'm not saying there is no role for this in gaming, I think it would be great if Nintendo could make it cheaply for the Wii and developers created games that could use it effectively... but that has been tried before in PCs... without much success.
Parent
Re: (Score:3, Insightful)
How about people just send him some money so he'll keep doing what he's doing and make it free?
Re:Nintendo! Hire Johnny Lee! (Score:4, Insightful)
Only problem now is that i can't see the TV
And besides, it's not what direction you're looking, it's what direction you're looking from. Move your whole body to the right while continuing to look at the TV and the display on the TV changes perspective. Not to mention depth of field, and distance from the TV. Did you even watch the video?
Why am I even responding to an AC comment?
Parent
Re:it's all research, man (Score:4, Insightful)
Parent
video down (Score:5, Informative)
Worth mentioning.. (Score:5, Informative)
The TrackIR solution linked above costs around as much as a Wii itself.
Fantastic (Score:5, Interesting)
Better than a medal (Score:4, Insightful)
Nintendo, are you listening?
Editors need to edit. (Score:5, Informative)
You spelled 'too' wrong.
Remind me again... (Score:3, Insightful)
Combine this with the weight-shifting capability of the Fit, and you've got an immersive gaming experience that's second only to the holodeck.
So. Freaking. Cool.
Johnny Lee Rocks! (Score:5, Interesting)
This will likely spur an avalanche of Wii hacks, and could easily cause wiimote sales to go thru the roof..
I'm totally enjoying the adventure Johnny!
Combinations? (Score:4, Interesting)
You don't need a Wii or the remote. (Score:3, Insightful)
Potentially you could just use a webcam with an IR filter in front of it instead of a Wii remote.
Note: 1) there is usually a filter to filter out IR inside most webcams, so that would have to be removed. 2) IR emitter tracking would have to be done on the PC instead of inside the Wii remote.
Re:You don't need a Wii or the remote. (Score:4, Informative)
Parent
HOLY SHIT (Score:4, Insightful)
Re:Aiming a gun by looking at your target (Score:5, Interesting)
Parent
Re:processing power (Score:5, Interesting)
Not to trivialise what Johnny is doing there is basically measuring the position of the wiimote in relation to the sensor bar - something it already does. The code to do this shouldn't be that difficult. The true genius was in him realising that you could do this easiest by reversing the moving component and the stationary component.
Apart from some smoothing algorithms, this is no more complex than reading the wiimote's pointer position and mapping that to a camera viewpoint.
Parent
Re:processing power (Score:5, Insightful)
It's so simple that you can do something with it, without having to wait for IBM, or Nintendo or any other big-$$$ company to bring out the relevant hardware in maybe 5 years.
Parent
Re:processing power (Score:4, Insightful)
Parent
Re:processing power (Score:5, Interesting)
Parent
Polarized light not compatible with LCD (Score:5, Informative)
They only work with DLP projectors (uses little mirrors), CRTs, plasma, and upcoming display technologies like Field Effect Displays and LED displays. Obviously there are a lot of display technologies that do work there, but LCD is a very popular technology for widescreen TV and of course, for PC monitors.
Either way you do it, you also have to double the grunt of your rendering system (or half your graphical complexity), and you need specific software support to get it right (you can go a long way with a driver that knows it's rendering for stereoscopy and just produces the correct eye POVs, but the glitches you get in the foreground and HUD are only tolerated by enthusiasts.). With shuttering you need glasses. With cross polarization you need to double the number of display elements (by having two displays or a special display with double the horizontal resolutions). Used in POV applications, all of these technologies are a one-user gig.
Stereo "Wii-D" will probably never happen ; half the audience have an incompatible display device, the system does not have an enormous excess of GPU grunt. Stereo3D would only be common with one of the following display devices...
* Personal head-mounted 3D display (probably VRD goggles)
* Large area wide aspect flatpanel displays with inherent stereo 3D support built in at the factory (which means basically doubling the vertical rez and making a special polarized filter for the screen).
The parallax effect that Johnny Lee demonstrates conveniently exploits the tendency of the human brain to "fill in the gaps" ; I'd be intrigued to see how convincing it really is.
As another poster points out, head tracking really isn't very well received for the PC, because the PC is an inherently static device. You can move your head, but your hands have to remain fixated on the keyboard / mouse. The Wii has an advantage here because the input device moves around with you. Several times during Zelda I got up from my chair and started moving almost involuntarily, my whole body was immersed in the game. I would never have tried that on the PC ; when I feel the urge there it probably just contributes to my neck tension.
If the static, 3rd person POV of Zelda can make this gamer rise up and move, a game armed with a head tracking linked POV would be compulsively immersive, even without stereoscopic 3D.
Parent