PDA Speech Translator

jlowery writes "Not quite as good as a babelfish, but a PDA that does translation is probably better than resorting to hand gestures alone. I could see this as a boon to the tourist who travels to places where English speakers are uncommon."

Could work, in a limited sense..

[iantri]

There is a program that already exists for the Palm (unfortunately I do not remember the name) that allows you rudimentary communication with one who speaks a foreign language by translating common phrases, selected by tapping on the screen.

I realize that this software is supposed to be somewhat more powerful, but what I am saying is that even limited translation programs are useful for tourists.

text

[Anonymous Coward]

As speech recognition technology gets better, and as handheld computers get more powerful, audio translators are becoming a more practical proposition.

Researchers from Carnegie Mellon University, Cepstral, LLC, Multimodal Technologies Inc. and Mobile Technologies Inc. have put together a two-way speech-to-speech system that translates medical information from Arabic to English and English to Arabic and runs on an iPaq handheld computer.

The prototype falls short of Star Trek's fictional universal translator in several ways. The system is not transparent -- it must be switched between Arabic-to-English and English-to-Arabic modes. It also works only when the speakers are talking about medical information, and it's only about 80 percent accurate in the lab.

The device shows that it's becoming possible, however, to provide automatic translation using a portable device. "It's good enough to make yourself understood," said Alex Waibel, a professor of computer science at Carnegie Mellon University and a founder of Mobile Technologies Inc.

The effort is one of a series of projects aimed at providing the armed forces with automatic translation for medical and force protection situations and making automatic translation in a wider set of subject areas available for tourists during the 2008 Olympics in Beijing, said Waibel.

The Speechalator prototype uses a built-in microphone and a language-selection button. "You push on the button on the iPaq and speak a sentence and then the translation comes out... in the other language," said Waibel. "You can switch it into the opposite mode when the other person answers and it translates back into your own language."

The software consists of three components: a speech recognizer, a translator, and a speech synthesis engine. "Each one of these components have slight twists to them... in order to work properly for speech translation," said Waibel.

The researchers modified the speech recognition engine to optimize it for handling spontaneous speech.

The translation system has the biggest twist. It extracts the key meaning from the input sentence and translates it to an interlingual, or intermediate representation, and the process depends on the speech being contained in a certain domain, or context, like medical information. "It's just certain nuggets in the phrase that... you need to extract," said Waibel.

The process is akin to constructing a medical-context template that fits the key information, then filling in the template, said Waibel. This process makes it possible for the system to handle spontaneous speech. "We go fishing for the nuggets," he said. But it is also a limitation -- the system must know what domain a speaker is talking about.

The researchers are working on a system that can handle multiple contexts and automatically switch between them, said Waibel. "It can, for example, recognize 'now you're in the hotel reservation domain', or 'now you're in the conference registration mode', or 'now you're talking about medical problem'," he said.

To come up with templates that handle different domains, the researchers collect a lot of data from people talking in those domains, said Waibel. "The more data we collect the better coverage of all the possible ways you could be saying [these things] becomes," he said.

The difficult part was fitting the software required to do two-way translation in the 64 megabytes of memory contained in the handheld computer, said Waibel. "You need two recognizers, two synthesizers and two translators to make [it] happen in both directions," he said.

The prototype also has a camera attachment that translates text like that on street signs, said Waibel. Snap a picture of a sign with the camera and it automatically extracts the text region, puts the text through a character recognition program, then translates it, he said. "What you then see on the screen is the picture of the scene with a sign and then underneath an English subtitle," he said.

Travelling

[elf-fire]

Well. I have been to quite a few places where English was not exactly lingua franca. In most of these places semi-right pronounciation of foreign words would not have had a big impact. Hand gestures and my favourite dictionary (which contains pictures of just about anything one would ever need 'on the road') have always been sufficient to find a hotel, a train or bus ticket out and some food. For the latter: Just walking into a restaurant's kitchen and pointing at the visible ingredients (dead or alive ;) ) suffices, and can generate a lot of fun in the process :)