Supporting Synonyms and Multi-Language Voice Input in Android

My Android Wear app, Resistor Decoder, takes voice input from the user. I wanted to support several ways that users could say the “same” thing while maintaining¬† architectural support for multiple languages.

Synonymous Inputs

In my app, people speak the names of colors that they see on a resistor. In English, most of the relevant colors have a single name, but a user could reasonably describe this color as either “violet” or “purple,” and the app should behave the same when given either name for what it thinks of as “violet.”

Android developers are already encouraged to use String resource files to tokenize Strings. The same basic mechanism works great for supporting synonymous inputs, with a slight modification.

The normal way to use String resources looks like this:


<string name="violet">violet</string>

You might look for “violet” in your input like so:

// assumes you have a list of Strings from
// your speech input
String violet = this.getResources().getString(R.string.violet);
// check to see if the next input String is the
// word for violet

Here’s my modification:


<string-array name="violet">

Now, you look for any of the Strings that mean “violet”:

// assumes you have a list of Strings from
// your speech input
for (String violet : myresources.getStringArray(R.array.violet)) {
   // check to see if the next input String is
   // one of the possible words for violet

Each word or phrase that your app supports is sorted into arrays with other synonymous words or phrases. Each <item> under a given array name will be treated as if it’s the same input by the app. I recommend doing this even for words or phrases that have no synonyms, since this may not be true in other languages.

In my case, my inputs are just colors, but in your case, maybe it’s, “show me the forecast”, “what’s the weather going to be”, “what’s it going to be like tomorrow” etc. You can easily expand this list as you get feedback from users, or possibly from Analytics, about other phrases your users want to say.

Multi-Language Support

Instead of producing translations for one-to-one mappings between tokens and strings, as you’d typically do, you now produce translations for your string arrays. Because each language will have its own words and phrases to express the concepts of your original language, you must provide your translators with special instructions here. They are used to providing a single translation for a single word or phrase; this is quite a shift for them.

For example, “orange” is the only reasonable way to say “orange” in English, but in Korean, there are three ways. If you provided your translator with your string array that mapped the token orange to the word orange, they may just provide a single way to say “orange” in Korean. Likewise, asking a translator to translate both “violet” and “purple” into their language may seem like a strange request if they only have one word for that color. Take care to provide clear instructions so that the translator understands the task.

Once this is done, the code that supports synonymous inputs “just works” and automatically supports the arrays provided for any other language. For example, my app supports one way to say “red” in English but three in Korean.


Play Store, Android Wear, and Internationalization

Here’s a quick internationalization tip for managing your Play Store assets for Android Wear apps:

1) Run your app in a Wear emulator paired with a physical Android device.

2) One by one, go to each “screen” in your Wear app that you want to show in the Play Store entry, and perform step 3 for it.

3) For each supported language, change the input language on your Android device (via Setup -> Language & Input)

4) Take a screenshot of the emulator window, and save it with an appropriate name.

5) Repeat #3 until all languages are represented for the given screen, and repeat #4 until all screens are represented.

The magic of this technique is that the Wear emulator will switch languages along with the Android device, so you only need to navigate through your Wear app once to get screenshots for all supported languages.

Amazon Fire Phone

Amazon just announced the Fire Phone, a mostly average phone running their broken version of Android. There isn’t much it offers that other phones couldn’t do with some third-party software installed, but here’s what I’m excited about.

As you can see, it has four front-facing cameras. Forget about why they put those cameras there, since it’s a gimmick, but here comes the cool part. If you’ve made video calls, you know that you can either choose to make eye contact with the other person by looking in the camera (which is nice for them), or you can look into the eyes of the person on your screen (which is nice for you), making it seem to them as if you’re looking at their throat. You have to make this choice because you can’t put a camera where the person’s eyes are without drilling a hole in your screen.

However, with four cameras, you can derive a virtual camera that is where the person’s eyes appear on your screen, meaning you can look into a person’s eyes on your screen and they see this happening, too. The only unknown is whether or not the CPU or GPU in a phone would have the power do to this.

Smart Doorbell

This is how cognitive surplus works:

I should do laundry… Nah, I’ll write an unnecessary software application instead.

The latest unnecessary software application is my Smart Doorbell program. Consisting of a python program and libraries, Smart Doorbell solves the age-old problem of how to replace cheap things with expensive things that have a few more features.

Summary from the project README:

This program turns your Raspberry Pi into a doorbell. It can replace a traditional hardware doorbell, and it integrates with openHAB and has a number of customization features.

App Permissions in Real Life

You’ve gone to the local county fair. You want to play the game where you throw darts at a balloon to win a prize.

First, however, you have to tell the vendor your phone number and give him the contact info of everyone you know. You also agree that the vendor can find out precisely where you are in the future, whether or not you’re playing the game.

Of course, this sounds so very reasonable, so you agree.

When you’re done playing, you walk away from the booth, but the vendor still has your phone number and the full contact info for you and everyone you know. If you decide never to go back, he won’t be able to know where you are anymore, so there’s that.

bitcoins, tulip bulbs, and houses

Bitcoins have been gaining in popularity. Recently, the news was that they had surpassed a value of USD$1000 per bitcoin. (Of course, bitcoin proponents might instead say that the USD is valued at 0.001 bitcoin.)

I’m not an economist, but I thought I’d share some graphs of the value over time of items that seem to appreciate in value beyond their utility. Can anyone spot any similarities to bitcoins?

Tulip bulbs (price information is spotty, but nobody contests the overall trend):

Tulip bulb prices over time

House prices:

House prices over time

And now, bitcoin prices:

bitcoin prices over time

I try to figure things out. Sometimes this leads to a thought. Sometimes I write it down.

Bad Behavior has blocked 92238 access attempts in the last 7 days.