BrainPort Lets the Blind "See" With Their Tongues

Hugh Pickens writes "Scientific American reports that a new device called 'BrainPort' aims to restore the experience of vision for the blind and visually impaired by relying on the nerves on the tongue's surface to send light signals to the brain. BrainPort collects visual data through a small digital video camera and converts the signal into electrical pulses sent to the tongue via a 'lollipop' that sits directly on the tongue, where densely packed nerves receive the incoming electrical signals. White pixels yield a strong electrical pulse and the electrodes spatially correlate with the pixels, so that if the camera detects light fixtures in the middle of a dark hallway, electrical stimulations will occur along the center of the tongue. Within 15 minutes of using the device, blind people can begin interpreting spatial information. 'At first, I was amazed at what the device could do,' says research director William Seiple. 'One guy started to cry when he saw his first letter.'" There is some indication that the signals from the tongue are processed by the visual cortex. The company developing the BrainPort will submit it to the FDA for approval later this month, and it could be on sale (for around $10,000) by the end of the year.

How about the back or chest?

[davidwr]

Sure, the resolution won't be as fine but it will be a lot less obtrusive to wear a sensor wrapped around your torso than to have something on your tongue with a wire sticking out of your mouth.

A practical version of that sensor net the blind lady wore on Star Trek back in the '60s will likely be on the market before 2067, assuming technology doesn't leapfrog it entirely.

Seeing Sound?

[Tablizer]

About 20 years ago I thought of a device for deaf people to "see sound" after reading that researchers have learned to read spoken words from from gray-scale sound spectrograms (frequency plots).

Now an off-the-shelf PDA or iPhone could probably do the trick of showing a plot with the right software. Some slashdot readers claimed it's too hard to learn if you never heard sound before. But it may be worth a try. Besides, some deaf people used to hear before an injury or illness. It's basically pattern-recognition, something humans are pretty good at given sufficient feedback.

Perhaps these devices can be combined and the frequency plots could flow through the tongue. However, I suspect there's insufficient resolution that way, and eyeballing it would be better. But, it's worth a try.

Re:Nothing to see here

[fuzzyfuzzyfungus]

The hardware seems like a fairly pedestrian evolution of cheap image sensors and high-density fabrication techniques.

The fact that the brain will, fairly swiftly, being interpreting electrical pulses on the tongue as visual input blows my insufficiently capacious mind.

Re:Seeing Sound?

[Anonymous Coward]

There are many people with sensory disorders that can see sound. Most of them don't like it but they're usually not deaf.

Re:Seeing Sound?

[ozydingo]

An iPhone or PDA easily has enough computing power to do a real-time spectrogram, but to be nitpicky, it's a time-frequency plot, not just a frequency plot. In my experience it's pretty hard to pick up the ability to read spectrograms of speech accurately and quickly, then again it's not my only access to speech. At the very least it would increase a deaf user's awareness of sound in his or her environment, and there would be at least a minimum level of discrimination between various types of sounds.

As for alternate modes of sensation (assuming something like a cochlear implant is a no-go), look into some of the work being done in vibro-tactile devices - http://techtv.mit.edu/genres/18-education/videos/3557-speakers-and-signers-ted-moallem-sensible-technologies----sensory-communication-aids-for-the-developing-world [mit.edu]

The Brain That Changes Itself

[smcdow]

You can read all about the work leading up to this device, why it works, amazing stories of recovery from brain injury, and other cool stuff in a book called The Brain That Changes Itself [amazon.com].

This is one of the best books I've ever read.

Danger... Hot Food

[fractalVisionz]

What happens when you burn your tongue? Does your "sight" degrade or get blurry while your taste buds are being repaired?

Useful for sighted people?

[Mprx]

Tactile reaction time is faster than visual reaction time. If the resolution is high enough and the switching time fast enough, could this system be advantageous where fast reactions are needed (eg. games, sport, driving, combat, etc.). Could it be combined with normal vision for a kind of minor precognition?

How about using it for extended vision with more frequency channels, wider or narrower field of vision, faster automatic brightness control, etc? Touch has multiple channels but how many are high enough resolution to be useful?

As anyone who's used psychedelic drugs will know, the human visual system is bottlenecked by the eyes. The brain can certainly handle more powerful sensors so we should be working on making them.

The inverse

[MathiasRav]

I'd love to be able to see tastes - that is, have my sense of taste piped to the brain as vision instead of taste. I wouldn't want my vision permanently replaced, but I'd love to experience the brain visualising what I taste. And hey, instead of the usual 3 dimensions [wikipedia.org] (or more if you're lucky [wikipedia.org]), you'd have 5 factors [wikipedia.org] to go with.

Re:Pattern Stream Processing.

[curmudgeon99]

Jeff Hawkins (founder of Palm) wrote an excellent book "On Intelligence" that deals extensively with pattern stream processing. I think the human brain is so plastic that it can handle much, much more stimulation than we give it credit for. Consider the contrast in the sensory ambient environment of the modern man versus the same man 100 years prior. Quite different. I think the human brain can do anything--and we are not at 1% of its capacity. If you want to add more inputs through an ankle pattern, I think your brain could learn to regularly monitor those inputs. It would learn its own language that you could read.