Robotic Hand Translates Speech into Sign Language 135
usermilk writes "Robot educators Keita Matsuo and Hirotsugu Sakai have created a robot hand that translate the spoken word into sign language for the deaf. From the article: 'A microchip in the robot recognizes the 50-character hiragana syllabary and about 10 simple phrases such as "ohayo" (good morning) and sends the information to a central computer, which sends commands to 18 micromotors in the joints of the robotic hand, translating the sound it hears into sign language.'"
How to sign First Post? (Score:3, Interesting)
You know what would really spoil those deaf kids is, instead of a robot doing sign language, a robot that shows images or words based on what a speaker says. I know, I know; creating a robot to do this is a feat within itself and impressive in its own right, but perhaps there are better ways of communicating with a robot if it can already perform more than adequate speech recognition.
Re:How to sign First Post? (Score:2)
Re:How to sign First Post? (Score:2, Informative)
Re:How to sign First Post? (Score:3, Informative)
Universal Sign Language (Score:2)
Yup, much the same as how, unfortunately, no one's come up with a universal spoken or written language. Gosh, let alone trying to get a universal programming language...
Re:Universal Sign Language (Score:2)
Hmm...isn't the middle finger signifying "Fuck You" pretty much universal?
I wonder if the robot hand translates that properly?
Universal Gesture Less than So (Score:2)
I know it's a joke you're making, but actually, I don't believe it is universal although it's rapidly spreading coupled with English. I want to say that most cultures have an "up yours" gesture of some sort involving a hand punching up with some kind of finger gesture, but that's probably my ethnocentrism speaking.
Re:Universal Gesture Less than So (Score:2)
Not really ethnocentric. The phallic symbolism is pretty much global. Though one or two cultures prefer to make a round shape, indicating a different set of genitalia are in play.
In rural Greece, showing the palm of the hand [ooze.com] (link covers many types of "finger") is the rudest gesture you can make. If you go there, don't wave
Re:How to sign First Post? (Score:5, Informative)
The relationship between a language & sign language does not work like that.
From the wikipedia sign language page [wikipedia.org] and You know what would really spoil those deaf kids is, instead of a robot doing sign language, a robot that shows images or words based on what a speaker says.
That doesn't really sound like a robot, but speech recognition software connected to a teleprompter (or monitor)
Re:How to sign First Post? (Score:3, Informative)
Re:How to sign First Post? (Score:3, Informative)
Sorta, but not quite. You can fingerspell words you don't know, and some words are derived from their associated lett
JSL vs ASL (Score:2)
The next question which I have is the significance of body positioning of signs in JSL. Most ASL signs have migrated to the face and upper-chest region, but I know some sign languages have a great amount of significance in the body positioning and it may ra
Re: Kanji and *SL (Score:1)
While in Japan I was able to learn some JSL at the hands of some of the top translators in the country (my failure to learn more was my fault, not theirs). I was surprised to find JSL
Re:How to sign First Post? (Score:1)
Re:How to sign First Post? (Score:1)
Re:How to sign First Post? (Score:1)
Re:How to sign First Post? (Score:2)
Which do you think is better ?
a)A robot that translates words into pictures
b)A robot that translates one language into another
Deaf people aren't stupid, at least no worse than the rest of us. You do realise that they probably already know sign language (which is the reason for the robot) so to be shown stupid pictures would be a little demeaning, don'
Re:How to sign First Post? (Score:1)
Re:How to sign First Post? (Score:2)
A syllable is defined as a single uniterrupted sound forming part of a word or, in some cases an entire word. (Example left out...)
We know that there are only 26 letters in the English alphabet. But how many different syllabic combination sare there? According to the late Pr
Scutters! (Score:2, Funny)
Additional warning:
Do not let this robot pat you on the back whilst near the top of the stairs.
Re:Scutters! (Score:1)
Re:Scutters! (Score:2)
But you must be protected. You must go down the stairs...
Re:Scutters! (Score:1)
Over Kill? (Score:5, Insightful)
Re:Over Kill? (Score:1)
Text is not the same (Score:3, Insightful)
American Sign Language is not English (American or other).
Thus, translating speech to ASL would reach people that that understand ASL but don't read Englih.
Re:Text is not the same (Score:1)
Re:Text is not the same (Score:2)
Re:Text is not the same (Score:1)
Having said that, the average reading age of the average adult isn't that great - for example http://news.bbc.co.uk/1/hi/talking_point/3166967.s tm [bbc.co.uk] cites a study that say "millions of adults do not have the skills of the average 11 year old" (implying that it's because their skill set is lower, not higher!).
So, certainly in the UK, though prof
ASL users and bad reading (Score:2)
Re:ASL users and bad reading (Score:2, Insightful)
But kids don't spontaneously pick up on writing. What you are asking these deaf children to do is learn ASL (which is basically not written), and the written form of a foreign language. That's like only hearing English (no writing) and only seeing written Chinese (no speaking). That's tricky to
Signing, but not reading (Score:3, Interesting)
It is my understanding that children can learn to sign before they can learn to read. (In fact hearing children can learn to sign before learning to speak.)
Similarly, developmentally challenged people, such as certain people with Down's Syndrome, never learn to read, but can sign just fine.
Reading takes certain specific brain functions, and it is not inconceivable that there
Re:Signing, but not reading (Score:2)
Re:Text is not the same (Score:1)
Re:Over Kill? (Score:5, Informative)
Because signing is the native 'tongue' for most deaf people - and it is easier for them to communicate using sign language (over text) - just as its easier for you to understand speech (over text).
Basically - the same reason that some British TV (and undoubtedly many other channels around the world) have a signer translating the news rather then scrolling text.
Meh - then just have video of someone signing... (Score:2)
I'm not saying it's not an interesting project, but it's not a practical solution to the problem.
More than just words (Score:2)
Sign languages tend to be more than just words. the positioning and motion of a sign conveys location and tense of nouns and verbs. It would be like speaking English without being able to conjugate any of the nouns or verbs.
They could, perhaps, dynamically generate pictures of the signs to convey more infor
OK, you're just being silly. (Score:2)
You're just being silly. All this robot does is take words and map them to gestures. It doesn't convey all this crap you're imagining. And if you're going to do signs which require relation to body parts - as many do - you're going to need a big f'ing robot body to make it visible to lots of people - and you're back to viewing from one direction.
In the very worst case scenario, they could have a 3d representation of a hand
Re:Over Kill? (Score:2)
I realize I may be a bit unusual, but I tend to understand text faster and more accurately than speech. It is certainly faster to speak than it is to write out the text, but I can read an entire paragraph in a second or two, whereas communicating the same information as speech would take much longer. Also, I usually visualize the speech as text in my mind and then "read" it, rather than interpreting the words directly.
Re:Over Kill? (Score:2)
For example sign for the English word 'worst' is better described as 'dramatic increase or change.' So, if I say 'Wow, my score on Asteroids has improved' I would use the signs 'wow, game, asteroids score worse' finger-spelling the word asteroids and having my eyebrows up and smiling as I sign worse. Also
Visibility? (Score:2)
blind and deaf, teaching, and other uses (Score:2)
The other use could be for teaching sign language. There's a lot of people that know a little sign language, but perhaps not enough to teach someone. Seeing a robotic hand do it in th
Re:Over Kill? (Score:1)
Re:Over Kill? (Score:2)
Does it also distinguish... (Score:2, Interesting)
I seem to recall that sign languages differ between countries, same as 'natural' language.
However this is really great for deaf people.
Re:Does it also distinguish... (Score:2)
However, going from Bahston to Hyooton, to Narlenz can be harrowing. Anyone remember the campaign trail with Bush version 1 was traipsing thru Louisiana? I can, a bit. I remember all that twisty, silky, unintelligible stuff that obviously was aired for entertainment value by the networks. (I am a person of less than a lighter color
Picture of robotic hand (Score:5, Funny)
More Useful As Software (Score:4, Insightful)
Deaf people could carry a PDA, and when they need to find out what someone is saying, they can hold the PDA up like a microphone, and watch the screen, assuming the translation is at least reasonable accurate...
Of course they could lipread too but some find that harder than others, and this could also be used eventually to cross language barriers?
I imagine it's extremely hard to lipread a foreign language.
it's about form factor (Score:5, Funny)
Yes but not nearly as intimidating. Who's going to get their lunch money taken -- deaf kid with a PDA, or deaf kid with a giant robot hand?
Re:it's about form factor (Score:1)
Parent is not kidding! From TFA:
!!!
Re:it's about form factor (Score:2)
Remember, though, that this is Japan. Kid with PDA probably merges with the Wired. Kid with part of giant robot merges with... well... pretty much everything, after a while.
Once that happens, your lunch money is the least of your concerns.
Re:More Useful As Software (Score:2)
> 3d hands with low microprocessor overheads...
There is no need for 3d. I know a woman who makes her living as an ASL translator. She spends most of her time sitting sitting in front of a monitor and camera wearing earphones.
The robot hand is pointless. The computer could just as well generate cartoon images.
Re:More Useful As Software (Score:1)
Yes, render (Score:2)
Why not just type it out to the screen?
Re:More Useful As Software (Score:1)
Amy Pretty (Score:3, Funny)
http://www.imdb.com/title/tt0112715/ [imdb.com]
Actually that's an interesting point. (Score:2)
If we consider the gestures as a series of movements produced by predetermined actuators (junctions), they can be quantized and stored in a vector, it's just numerical input, and could be classified as a different kind of speech.
Training the a gesture reader would be equivalent to searching inside a soundwave database (find the closest match, reject if there's any sig
Re:Amy Pretty (Score:2)
Say the wrong thing and you might be "bitch-slapped". Or, with that alloy, maybe beaten to a shallow tallow pulp... (woo... any onomatopoeia coming from that hand, a la "AKA I'M BATMAN!!!"?... squish, khoomp, BAM, POW, THOONG...
But, certainly, the Japanese model might actually USE the middle finger gesture. Over there it is NOT the same thing as here or in some English-speaking countries. And, the Japanese arm/hand most likely WON'T be waving "come here
I've gotta hand it to these scientists (Score:3, Funny)
So it's not just a hand, but a hand with two legs!
Other uses? (Score:1, Funny)
Re:Other uses? (Score:1)
This gives a whole new meaning (Score:5, Funny)
Re:This gives a whole new meaning (Score:1)
A different dept. (Score:1, Insightful)
Wonderful (Score:1)
This is great and all... (Score:1)
You can talk to the hand, sure but, that doesn't help you read the deaf persons hands..
--
In retrospect:
The article makes no sense (Score:3, Interesting)
Re:The article makes no sense (Score:2)
Universal Signs (Score:2)
It would be interesting to know how these motions translate, if at all.
Re:Universal Signs (Score:1)
Re:Universal Signs (Score:1)
Re:Universal Signs (Score:1)
what if (Score:1)
Japanese Sign Language link (Score:2)
Re:Japanese Sign Language link (Score:1)
Re:Japanese Sign Language link (Score:2)
-- You're on crack, and drooling
-- You snorted Vegemite while hiking at YO-seh-MIGHT (Yosemite), but ended up (butt-up) a sulphur spring, except it had Killer Sulphur like in parts of Japan
-- You got whacked aback of the head with a finely-tuned, perfect-pitch, exquisitely-balanced duratanium rod, and freeze frame caught your eyes before they popped out
-- You're on your back, approaching orgasm, and drooling (just before being hammered by a duratanium rod, totally
Recognziing Sign Language (Score:5, Interesting)
ASL isn't like English in that there are always specific words- a lot of it has to do with spacial context (where in the signing space the sign was made) and a whole class of signs that don't translate directly into words (they're hand shapes which can translate into an event or a description of an object or set of objects).
And, as the research page shows, facial expressions and even facial movements can be part of a sign.
Of course, this is American Sign Language, Japanese Sign Language may be very different.
Re:Recognziing Sign Language (Score:1)
Actually, AFAIK facial expressions, body tilt etc. have a para-linguistic meaning, much like the tone of voice and facial expressions in hearing communication.
Therefore, they are not - strictly linguistically (structuralistically) speaking, part of the sign itself, but rather a part of the co-text and context.
Anyway, that's what I remember from that one lecture a year and a half ago...
Re:Recognziing Sign Language (Score:2)
They have para-linguistic meaning but they're an important part of signining. It's not equilivant to the tone of voice as it would appear in a Western language (English, French, German), but it's often part of the sign itself in that certain facial expressions or mouthings should accompany certain signs.
They're also *vital* for things like questions, where eyebrow position are the indicator that the statement is a question. translated to English, it might be more akin to tone: "Going
Not only paralinguistic (Score:2)
For instance:
- negation: in ASL the manual marker for "NOT" is optional. The associated facial expression is mandatory. If the manual sign is omitted, the facial expression must extend over the clause being negated.
- temporal: eye aperture can indicate whether an action took place in the past. T
Wait... (Score:1)
this has been done dozen of times (Score:2)
Very large? (Score:2)
Been There, Done That... (Score:2)
There is an article Evolution of Mechanical Fingerspelling Hands for People who are Deaf-Blind [stanford.edu] that talks about the development of this technology since 1977.
There are a couple of challenges with this type of technology. Sign language does not depend only on finger movements but gestures and facial expressions to convey emotion and context. Finger-spelling hands, being mechanical, can only accept data so fast before they start "choking" and sezing up/breaking (we tried hooking one up to a teleprompter ap
Not the first... (Score:2)
On another note, this sort of translation is actually more difficult than a voice-to-text, text-to-sign translation. As someone
Can't see?, you can feel the fingers... (Score:2, Informative)
Re:Can't see?, you can feel the fingers... (Score:1)
That's Cool (Score:2)
MST3000 (Score:2, Funny)
Why didn't they just use a monitor with 3D image? (Score:2)
Great! (Score:1)
interesting but how practical? (Score:2)
guess this is more for the sake of doing rather than being practical
It's cool, but... (Score:1)
Why use a robot? (Score:2)
Oh for heaven's sake... (Score:2, Insightful)
Instead of trying to analyze these achievements in the rather constricted mould of "Why not 3D graphics" or "Why not text on a screen", consider the use of this technology in the future - when say, the robots to help disabled people finally get off the assembly lines. By then, this process would've been refined to the point of being able to do an excellent job in communications.
As a researche
I bet deaf people hate it (Score:3, Insightful)
sigh.. incorrect Japanese (Score:1)
How smart..? (Score:1)
I was an interpreter for about 7 years (Score:2)
I remember once telling a story, having given place names in space
Re:Talk to the hand (Score:1)