Robotic Hand Translates Speech into Sign Language

usermilk writes "Robot educators Keita Matsuo and Hirotsugu Sakai have created a robot hand that translate the spoken word into sign language for the deaf. From the article: 'A microchip in the robot recognizes the 50-character hiragana syllabary and about 10 simple phrases such as "ohayo" (good morning) and sends the information to a central computer, which sends commands to 18 micromotors in the joints of the robotic hand, translating the sound it hears into sign language.'"

Re:How to sign First Post?

[tpgp]

Good lord! I imagine the Japanese language with its 1945+ character alphabet is hard enough to learn; learning Japanese sign language must really suck.

The relationship between a language & sign language does not work like that.

From the wikipedia sign language page [wikipedia.org]

A common misconception is that sign languages are somehow dependent on oral languages, that is, that they are oral language spelled out in gesture, or that they were invented by hearing people

and

On the whole, deaf sign languages are independent of oral languages and follow their own paths of developmental. For example, British Sign Language and American Sign Language are quite different and mutually unintelligible, even though the hearing people of Britain and America share the same oral language.

You know what would really spoil those deaf kids is, instead of a robot doing sign language, a robot that shows images or words based on what a speaker says.

That doesn't really sound like a robot, but speech recognition software connected to a teleprompter (or monitor)

Re:How to sign First Post?

[Anonymous Coward]

I believe it's a joke... but just to point out, there's ASL and BSL and...

Re:How to sign First Post?

[Jugalator]

Unfortunately, no [wikipedia.org] ;-)

Re:How to sign First Post?

[lewp]

Japanese has a whole bunch of kanji, but the various words in the language can be formed from a much smaller (hiragana, mentioned in TFA) character set that represents the various syllables in the words. These syllables are always pronounced consistently, unlike languages like English where sometimes it seems like nothing is consistent (and I'm a native speaker). Thus, the first thing that came to my mind was that teaching a robot spoken Japanese is probably quite a bit easier than teaching one English (though neither is a trivial task, obviously).

I know nothing about Japanese sign language, and practically nothing about American sign language, but I believe American sign language shares a similarity to written Japanese in that there are signs for common words most any competent signer knows (similar to kanji), and any particularly uncommon words can be signed out with the letter (or in the Japanese case, hiragana syllable) signs. Thus, I doubt teaching a robot enough Japanese sign language to be understandable wouldn't be any harder than teaching a robot American sign language. Which, once you've turned the speech into letters/syllables in the speech recognition part and programmed in the gestures, would be pretty much trivial. Japanese children's TV and manga aimed at kids (I'm told) mimmicks this behavior by mixing the simple kanji school children will have learned at a young age with the hiragana for the words that aren't expected to be known.

I'm shooting from the hip here based on what little experience I have with this stuff, so feel free to correct me, experts.

Re:Over Kill?

[tpgp]

Call me culturally insensitive but, why not simply translate speech to text?

Because signing is the native 'tongue' for most deaf people - and it is easier for them to communicate using sign language (over text) - just as its easier for you to understand speech (over text).

Basically - the same reason that some British TV (and undoubtedly many other channels around the world) have a signer translating the news rather then scrolling text.

Re:How to sign First Post?

[magefile]

I know nothing about Japanese sign language, and practically nothing about American sign language, but I believe American sign language shares a similarity to written Japanese in that there are signs for common words most any competent signer knows (similar to kanji), and any particularly uncommon words can be signed out with the letter (or in the Japanese case, hiragana syllable) signs.

Sorta, but not quite. You can fingerspell words you don't know, and some words are derived from their associated letters (i.e., one of the possible signs for "what" looks a lot like a "W" snapped into a "T", and one of the signs for toilet looks like a shaken "T"), but some of these are frowned upon culturally (cultural baggage due to decades of surpression of sign by hearing people). Too, if you depend on fingerspelling too much, you'll find it difficult to communicate; you won't be able to receive well, and while Deaf will put up with receiving it if they know you're learning, it's not sign language, and everyone knows it. Doesn't fit in all that well with the syntax and grammar of ASL, either.

Can't see?, you can feel the fingers...

[eMpTyBeitler]

In the early 90's I worked with the robotic finger spelling hands called "Dexter" & "Ralph". Those devices were intended for individuals who are both deaf and blind. An individual with this kind of disability must rest their hand on the back of someone's hand (or on the back of the robotic hand) and feel letters as they are signed by the hand/fingers.