Wiimote as Multi-Touch Display Controller 107
Tmack writes "While hard-hacks with the Wiimote are somewhat old news, this particular implementation is quite interesting. Using the infrared camera on the Wiimote, pens with LEDs instead of ink, and an LCD projector, Johnny Chung Lee of Carnegie Mellon University has created software to use them as a (relatively) cheap multi-touch display. Any surface onto which you can project becomes an interactive multi-touch display, as demonstrated in the video at the link. He has the software available for download, along with some other neat projects.
Lee has also documented another impressive Wiimote hack.
I can get one now? (Score:3, Funny)
Oh...the Wiimotes are out of stock too? *walking away, hanging head*
Re: (Score:2, Funny)
sure you can... Re:I can get one now? (Score:1)
I presume you're making a joke on this part, and you've been accordingly modded +2 funny. But I thought I'd mention that by my own experiences, the wiimotes have been consistently well stocked, even with the wiis sold out constantly. Apparently it is nowhere near as difficult for nintendo to make the wiimotes as is it to make the wii itself...
Re: (Score:3, Insightful)
either
(A) the Wiis are used less for multiplayer than Nintendo expected
or
(B) the Wiimotes have a higher survival rate than Nintendo expected
or
(C) A+B
Re: (Score:2, Funny)
Since this could imply that perhaps Nintendo expected more people to (inadvertently) send their wiimotes crashing through their TVs or sailing across the room...
Re: (Score:2)
I have to admit, I'm quite annoyed with the detection on mine. I keep wanting to bash the thing against my coffee table because it won't detect that it is pointing at the screen.
Re: (Score:1)
I have to admit, I'm quite annoyed with the detection on mine. I keep wanting to bash the thing against my coffee table because it won't detect that it is pointing at the screen.
Unfortunately, I cannot relate to this, as I have not been able to buy a wii for myself thus far. Your comment does remind me of my days with the good ol' NES, however. I know I intentionally swung those controllers around by their cords and bashed them on the floor for not working correctly.
I know there were many times where poor Ryu Hayabusa fell into one of many bottomless pits because the timing just wasn't right for the A button on those controllers. Even the "slow motion" feature on the "NES A
Comment removed (Score:4, Funny)
Re: (Score:1)
Re: (Score:2, Informative)
A few notes if you're having trouble:
First, make sure your IR sources don't confuse the wiimote. Example: Opening my drapes on the window behind the TV on a warm, sunny day, made the wiimote useless.
Second, set the proper height for the "sensor" bar in the Wii console itself. It does make a difference, especially if you have a large TV
Third, make sure your batteries are properly charged. Don't wait for the warning to go to red. When it gets to 1 bar, put the batteries in the recharger.
T
Re: (Score:2)
The "IR Pollution" is the biggest factor affecting the performance of your wiimote, in addition to distance.
It also affects the ability of a TiVo to control an external tuner like a cable box with IR properly. Even if the cable box cannot see the source of the IR pollution, if the TiVo can see it, it cannot send the signal to change channels until that pollution goes away (IR bus busy).
So be careful how you set up your Wii sensor bar with respect to any TiVos that control external tuners with IR signals.
Before the Wii, another source of this problem was wireless IR repeaters that would turn any radio noise they
Re:sure you can... Re:I can get one now? (Score:5, Informative)
Check that there are no bright IR light sources nearby. Sunlight and Christmas lights are two common ones that confuse the Wii Remote. Also make sure that your Sensor Bar is far enough forward that it isn't blocked at certain angles.
An easy way to figure things out is to go into the Wii Remote settings screen. There's a black and white image there that effectively shows you exactly what the Wii Remote "sees". You should see a few dots that do not flicker or go away as you move the remote around. If the dots disappear at any point during your tests, you may need to adjust your sensor bar.
If the lights appear to be working okay, but the remote is still confused, try turning down the sensitivity of the remote. That will encourage the remote to ignore light sources other than the sensor bar.
Re: (Score:2)
Actually, my problem is how close I have to be. They should make a smaller "nearsighted" bar.
Since I got my replacement contact lenses (and can focus at larger distances), it's better, though still not perfect. They need to make a sensor-bar mini, that's half the width, designed for closer use.
Re: (Score:2)
Re: (Score:1)
Re: (Score:2)
Ah, I see how that could be a problem. The minimum distance is really about 4-5 feet from the screen. Even then, you pretty much need to be in a chair to play effectively.
Possibly the best advice I can give you is this: Get a bigger TV.
Re: (Score:2)
Re: (Score:2)
If your TV is big enough and you're standing at least 4 feet away, then its probably your sensor bar. Either there is too back background IR light, or the bar is becoming invisible at certain angles. If it's the latter, you can try one of two things:
1. Move the sensor bar closer to the ed
Re: (Score:2)
Re: (Score:1)
Imagine a cone growing out from the front of the Wiimote. The further you are away from the sensor, the more likely that the sensor is seen within this cone. The closer you get, the more likely the sensor is outside of the cone.
For most pointing applications, the Wiimote only needs to see one end of the bar or another. Some applications might use the sightings of th
Wii Menu and Wii Play are roll-sensitive (Score:2)
Re: (Score:2)
One question is whether these apps require sighting the two points or not.
They can also get the rotation data from the accelerometer.
Can you experiment with covering up one end of the sensor bar to see if those things still work?
Re: (Score:1)
One question is whether these apps require sighting the two points or not.
They can also get the rotation data from the accelerometer.
Can you experiment with covering up one end of the sensor bar to see if those things still work?
Both. In Wii Menu, when I use both emitters, the hand cursor responds immediately to roll. When I use only one emitter, the hand cursor responds much more slowly, as if it is trying to filter out accelerometer noise. If I cover up one emitter on Wii Play's menu and then point the Wii Remote at one emitter, the cursor doesn't show up until I uncover the second emitter. In Pose Mii, if I cover up one emitter, my Mii responds to roll but spins in place until I uncover the other emitter.
Re: (Score:1)
They've made a secret deal with Sony. Sony doesn't compete with them in the console business, Nintendo creates an artificial demand for new televisions. Win-win.
Re: (Score:2)
Re: (Score:1)
I suspect that the original poster may be in a different country to yourself.
I live in the UK and have bought both a Wii and a Wiimote in the last week. The Wii was hard to get hold of, but nowhere near as hard to get hold of as the Wiimote. Eventua
Re:I can get one now? (Score:4, Funny)
Re: (Score:1)
Re:I can get one now? (Score:4, Informative)
=Smidge=
Re:I can get one now? No problem! (Score:1)
Wiimote is a Wii-Detector (Score:5, Informative)
Re:Wiimote is a Wii-Detector (Score:5, Informative)
they have size... (Score:3, Informative)
Re: (Score:1)
Re:they have size... (Score:5, Informative)
I was abit unclear.. (Score:2)
Re: (Score:2)
I'm doubtful the Wiimote uses any information from the bar other than yaw. Calculating distance to the screen is perhaps possible but it would have to account for things like not being pointed directly AT the bar (perspective skew)
Re: (Score:2)
True.. (Score:2)
But using just x/y is just fine, and a lot more precise..
Motion capture? (Score:1, Insightful)
Re: (Score:2)
My project (Score:3, Interesting)
Re: (Score:2)
Re: (Score:3, Funny)
But you can make IR patterns (Score:3, Insightful)
Using the motion sensors to answer calls [engadget.com] might be a bit awkward (some minutes into the clip).
Re: (Score:2)
It's actually rather simple...
Detecting Parkinsons Tremors (Score:5, Interesting)
Re: (Score:1)
Nothing new... (Score:2)
Re:Nothing new... (Score:4, Insightful)
And considering the application he offers is free this can only encourage more experimentation that can lead to even more innovative applications of a relatively cheap and abundant product which is (Nintendo sanctioned or not) becoming more and more of a multi-purpose tool.
Re: (Score:2)
Actually it's alot better than most products on the market atm. Mainly because of the speed, it's insanely fast compared too some commercial options that only support one touch.. And if you look at many of the multi touch interfaces being done you will see that they almost always lag 0.1s or something...
Re: (Score:2)
I think your just a negative Nancy.
Better than the last one (Score:3, Insightful)
Converted Candy Pen to IR Pen (Score:1)
Re: (Score:2)
We're going to try this in the office tomorrow. We would have done so today, but for some reason there were no Wiimotes around
Giant digital canvas (Score:2)
photoshop anyone? (Score:2)
Re: (Score:2)
-Rick
Re: (Score:1)
Doesn't work.. (Score:2)
Pretty neat! (Score:2)
I'm still waiting for the large-format flexy-screens that are as cheap as rolled paper. I love my LCD screens but they're just too damn expensive and fragile right now. I want 100" w
Re: (Score:2)
It looks like an insanely cool hack, and using things you have lying around the house too.
Re: (Score:2)
-Rick
Re: (Score:2)
Re: (Score:1)
How about using an IR laser pointer from behind the audience? the dot reflects the IR, the Wiimote picks it up.
maybe a combo green/IR pointer, with the IR is switched separately from the green. That way, the green dot is your cursor. (otherwise you're clicking blind)
But yeah, I love this. inset an LCD display into a desk, and have a mount for the Wiimote at the back of the desk. Who needs a mouse? (hm
More of a mashup than a hardware hack (Score:1)
Re: (Score:1)
Have you truly never used both hands at once for a task?
Re: (Score:1)
Re: (Score:2)
That is only true when the motion of both hands are completly independent of each other. Human brain is doing quite fine with using both hands at once for a single task (trying typing with a single hand or opening a bottle with just a single hand).
### I think the usefulness of multi-touch displays is limited.
Go and watch this demo video [youtube.com], now with a Wiimote you are limited to four points so many stuff seen there won't work, but al
Re: (Score:1)
About opening jars you are definitely right, but about typing I have to disagree. If you are a halfway decent typist you have the layout of the keyboard memorized. A multi-touch display is only useful if the content it displays changes (making it impossible to
Re: (Score:2)
I suggest you go and watch the demo video [youtube.com], it really gives some extremely simple and powerful example uses of multitouch and also makes it clear why that wouldn't be possible with a indirect pointing device such as a mouse. Look especially at the rotation and zooming and you will understand that there is no need to search for an object for both fingers.
Usefulness of multi-touch displays (Score:2)
It also might be interesting to use for zooming purposes. You can see the big picture as well as a box showing a portion of the big picture zoomed in to work on details...but moving that portion to where you want can be a pain in a lot of programs. While I couldn't write with two pens I think I could accurately move my one hand to the general area that I want to work on and hold it there or slide it occasi
Re: (Score:1)
Now, since that is far from common, I'll address the real issue with your post: limited creativity. Humans use their hands for multiple purposes quite often. Hands are actually designed for independent use. Your example of rubbing your stomach and patting your head at the same time is more of a question of how the arms are designed to work in tandem, this is more of a balance-oriented featu
This is pretty cool. (Score:1)
Re: (Score:1)
what isnt mentioned.... (Score:1)
* it can not distinguish between the different pointers
* wiimote can only track up to 4 sources
* nothing can get inbetween the line of sight of a pointer and the wiimote
if you watch the video carefully the author carefully does not cross his arms, this is because if he does he will line of sight (LOS in wow speak) the other pointer.
a fix to this could be to put the projector below a table and project upwards (inverted) on a g
Re: (Score:1)
point taken on the other issues though.
Re: (Score:2)
a fix to this could be to put the projector below a table and project upwards (inverted) on a glass surface so there is no change of LOSing any of the pointers.
I actually thought that was where he'd go with it.
Re: (Score:2)
3d touch (Score:2)
Normally I just nod and appreciate these hacking projects from a distance, but I think I'm going to jump in and do this for myself. Wish me luck, everybody.
Re: (Score:1)
Each point in each sensor corresponds to a known ray in space.
To find the point, you need to know which points correspond to
each other from the different views (this is easy if there's
only one point in each view), and then you calculate the midpoint
of the line that's perpendicular to both rays.
There's lots of research on this topic. For really good results,
you might want to investigate the Kalman filter. It's a
predictor-corrector algorithm that a
Re: (Score:2)
whose relative position is known. This what the TrackIR does.
Does it run linux? (Score:2)
Applications!!! (Score:1)
If this code were ported to Linux (which it will be soon) and you were using MPX, [unisa.edu.au] you could have a multi input computer!
The thing about this is that now there is ONLY software standing between me and this! [perceptivepixel.com]
Similar PiiMote/PyToy (Score:2)
1) IR webcam
2) IR-pen, similar to this guy's
3) software in python to track the IR light
4) demo-apps in pygame (tracking, swinging, 2 player pong)
It worked pretty well to be honest. I also atached some LED's to a wireless mouse and controlled the mouse from distance (similar to but not as good as the wii-mote).I tried to add multiple inputs using one cam by using color video input and color LED's instead of IR LED's, but then sc
Re: (Score:1)
I'm currently (and very slowly) working on an FTIR multitouch interface, similar to the stuff at NUI Group [nuigroup.com] but haven't gotten to the software part yet. They have libs there, but I'd be very interested in seeing your python stuff, as I'd much rather work in Python
Also, the IR pen setup should be quicker to build than the FTIR setup. I was especially impressed when I first
Re: (Score:2)
Re: (Score:1)
Seems like most of what I found is geared toward snippets, or full source code management. I wonder if anyone else here knows a good place?
Don't worr
Re: (Score:2)
Re: (Score:1)
As for figuring out how it works - well, that just takes time, reading,and thinking; just how much is yet to be determined
Re: (Score:2)
Re: (Score:2)
I released the code under GPL.
it's actually 3 files:
1) a library
2) a demo
3) a misc-lib to calculate the fps
the code is ver messy so let me know when you get stuck and let me know what you do with it
done it (Score:1)
Re: (Score:2)
2) i hate 4 point orientation, i need to tear apart his source so i can make it read 9 points as opposed to 4.
Re: (Score:1)
multiple wiimotes for higher res (Score:1)