Follow Slashdot stories on Twitter

 


Forgot your password?
Close
typodupeerror
×
Input Devices GUI Music Entertainment

Transforming Any Flat Surface Into a Control Panel With Sound 53

Posted by timothy from the scritcha-scritcha dept.
New submitter brunozamborlin writes "I just published a short video that shows how a very cheap contact microphone can be used to recognize different types of fingers touch and transform any surface into an interactive board. In the video we put the microphone over different surfaces such as kitchen tables and balloons and through realtime gesture recognition we show how we can play different virtual music instruments using a technique called physical modeling . A mobile version would be definitely possible." The project's Web page shows several more examples. Update: 12/31 15:17 GMT by T : Bruno Zamborlin points out that the surfaces don't need to be flat; instead, they simply need to be rigid.
This discussion has been archived. No new comments can be posted.

Transforming Any Flat Surface Into a Control Panel With Sound More

Transforming Any Flat Surface Into a Control Panel With Sound

Comments Filter:

  • Huh? (Score:2)

    by frnic ( 98517 )

    I don't see that "recognition" is achieved beyond being able to tell that touches were made and were "different" from each other - what those differences were was not demonstrated.

    • was coming out... then we go to the theatre and its muppets in space crossed with some kind of saturday morning cartoon reject.

    • Comment removed based on user account deletion
      • Comment removed based on user account deletion

        • Look for Acoustic Pulse Recognition. It's pretty well exactly what you just said.
          It uses a series of microphones, usually four of them on a glass pane, to locate contact.

          I would personally love such a setup for a glass desk top or keyboard tray, with the letters and buttons lightly etched into the glass. I'm sure creating imperfections in the glass would require a more rigorous calibration, but I believe it would be well worth the trouble.

          I can also imagine using more than four microphones would allow for v

    • Re:Huh? (Score:4, Interesting)

      by brunozamborlin ( 2542970 ) on Saturday December 31, 2011 @10:10AM (#38548020)
      In the video we see that different gestures are mapped to different sounds. For example the coin enables a certain abstract sound whist the bare fingers enables the bass sound and the nails enables the pad... this is done thanks to gesture recognition techniques (a modified version of Hidden Markov Models to work in realtime). Apologies whether it is not clear enough in the video though...

  • Training? (Score:5, Interesting)

    by Mabbo ( 1337229 ) on Saturday December 31, 2011 @09:42AM (#38547788)
    Do you have to give the system training for the specific purpose and gesture? And could the microphone be on the opposite side of the surface? If so, I think I've got an awesome new way to unlock my door.

  • I assume that things that sound the same in this demo, would be recognized as the same gesture.
    It is shown that doing the same gesture produces the same sounds over and over again, so that gesture would be reliably recognized.

    But, this would be extremly dependant on the type of surface and the spot on the surface, that the gesture is performed on. Changing either the spot on the surface, or to another surface entirely, would alter the outcome of the gesture, even though it is actually the same. That is also

    • Re:Flawed, or useable? (Score:4, Interesting)

      by Andy_R ( 114137 ) on Saturday December 31, 2011 @10:00AM (#38547944) Homepage Journal

      It depends what you want to use it for - as a replacement for a touch screen, it's probably not very useful because location information isn't going to be as easy to capture as timbral information, but as a musical instrument (or as a way of adding more sensitivity to existing electronic instruments, it's amazing.

      I wonder if accurate position information could be captured by triangulating with 3 of these contact mics?

      • Re: (Score:2)

        by Andy_R ( 114137 )

        Quick back of an envelope calculation... speed of sound 340m/s, sampling rate of most D/A converters 44,100 samples/sec... 7mm accuracy (or better if the speed of sound in the solid object is lower)? Not great, but good enough to turn any solid object into a virtual keyboard.

        • The speed of sound happens to be much higher in solids. The more rigid the material, the faster sound will travel through it.

        • Quick back of an envelope calculation... speed of sound 340m/s,

          In vacuum. In solid objects sound is typically an order of magnitude faster -- e.g. sound travels at approx 3400 m/s in wood.

        • Re: (Score:2)

          by PPH ( 736903 )

          The speed of sound is much higher in rigid objects. But this has already been done (to a certain degree) with high end electronic drum kits that can vary their response based on the location (radially) at which the pad is struck.

          They probably do some hardware DSP at much higher sample rates than 44K samples/sec. But this pushes the solution out of the realm of using a simple mic or two and a PC audio card.

      • It doesn't appear that the point of this project is to deal with positional information at all, but that it's a musical project where the type of contact the person does with the arbitrary surface is simulated as perturbing some physics-based audio source.

        Multiple-mic positioning has been done before in other projects.

      • I'm pretty sure with just two mics a fixed or known distance apart, you could get 2d coordinates. Just have to look at the timing offset of similar waves and calculate the radial intersection. On the flip side though, small CCD elements with sufficient resolution for a Wii/Smoothboard or Kinect style setup are more than cheap enough to use and relatively compact. A single mic as demonstrated might be useful for broadening the number of unique touch actions though.

        • Two mikes give you the difference in distance to the source, if you are doing it just with timing. The solution to a point that is x cm from one mike and x+y cm from another is a hyperbola. Done with radio, and the other way around, you get Loran navigation.

          Three mikes can intersect two hyperbolas for a location.

          Positional accuracy is limited. Human hearable frequencies top out at about 18 KHz, which in air has a wavelength of about 1.9 cm. At CD audio speed, you can get 44,000 samples per second, which

    • In this system the user can define a gesture by performing it just once. So users can easily define their gestures for every surface in few seconds. The next step would be to automatically recognise that the surface has changed so to avoid to re-train the system for every surface. Thanks for the interest!

  • So.. (Score:1)

    by Anonymous Coward

    Someone hooked up a microphone... Come on... Now 3 of these to track position may be news...

  • http://en.wikipedia.org/wiki/Theremin [wikipedia.org]
    Seems like a different input technique...

    Phil

  • Don Cherry has one of these (Score:4, Funny)

    by cangrande ( 199946 ) on Saturday December 31, 2011 @10:24AM (#38548132)

    http://youtu.be/qdJp5-g69go [youtu.be]

    [spoiler: it's an amusingly-dubbed video of eccentric Canadian hockey announcer Don Cherry, who wears really loud clothes and makes vigorous table thumping gestures]

  • Acoustical modeling to determine the point of origin of sounds is nothing new, and although it's a wonderful idea to implement it as they are here, it'll go to hell the moment there are other vibrations on the surface. A low-pass filter should stop most environmental noise bouncing off the table/surface from triggering it, but if you put down your coffee mug on your desk, or bump your leg to the table, you'll likely get false input. Not to mention, as others have pointed out, the processing costs. This c
  • Years ago, before youtube was really popular, I saw a video of a man doing a similar thing but couldn't find it recently!

  • I see, its a less useful version of this (Score:4, Informative)

    by james_van ( 2241758 ) on Saturday December 31, 2011 @12:00PM (#38548890)
    saw stuff like this years ago. cant find the exact tech demo that i originally saw that used 3 mics for triangulation, but this video is pretty close. even mocked up a rough version (low precision, slow response time, but pretty damn good for an afternoon worth of work and very little experience with arduino) tied to a projector in my shop. customers had a blast with it, thought it was the coolest thing in the world. http://www.youtube.com/watch?v=CxDHiwheK8w&feature=related [youtube.com]

  • I have some suspicions about this. They're getting more information out of one microphone than is usually possible. You might be able to extract some positional information by picking up the echoes off the edges of the object.

  • Hmm. I wonder how accurate of a keylogger could be made with something like that...

Slashdot Top Deals

In case of injury notify your superior immediately. He'll kiss it and make it better.

Close