CODEWITHSUNDEEP
javaandroidspeech-recognitionpocketsphinx
Guillaume LEFEVRE
Guillaume LEFEVRE

Reputation: 86

How to add phonemes recognition with pocketsphinx on Android

I'm working on a project about Cued Speech (it's a visual system of communication used with and among deaf or hard-of-hearing people). It is a phonemic-based system which makes traditionally spoken languages accessible by using a small number of handshapes, known as cues, (representing consonants) in different locations near the mouth (representing vowels), as a supplement to speechreading.

I already followed the tutorial about phonemes recognition (pocketsphinx_continuous), pocketsphinx on android and it's working. I'm working for now on Android but the final goal of this project is to make it work on Google glass. The best case is to write on the device the speech (closer as real time) and an avatar in 3D which will reproduce the correct hand configuration and lips mouvement according to the phoneme identified. (That's why i need to work with phonemes and not the word it self)

I read http://cmusphinx.sourceforge.net/wiki/phonemerecognition but i'm quite confuse about how to create my own "files" for french phonemes to make it works on Android.

What are the steps ? What kind of file should i create ?

In other words, is it possible to have the same result of pocketsphinx_continuous with phonemes on Android with voice input ?

I hope you can guide me !

Guillaume

Upvotes: 0

Views: 2254

Answers (1)

Nikolay Shmyrev
Nikolay Shmyrev

Reputation: 25210

Latest pocketsphinx demo includes phonetic recognition mode activated with "phones" search, see the source for details:

https://github.com/cmusphinx/pocketsphinx-android-demo

To use it with French you need to update it with French acoustic model:

http://sourceforge.net/projects/cmusphinx/files/Acoustic%20and%20Language%20Models/French%20F0%20Broadcast%20News%20Acoustic%20Model/lium_french_f0.tar.gz/download

And with french phonetic language model:

http://sourceforge.net/projects/cmusphinx/files/Acoustic%20and%20Language%20Models/French%20Language%20Model/fr-phone.lm.dmp/download

Like this:

    recognizer = defaultSetup()
            .setAcousticModel(new File(modelsDir, "hmm/french");
            .setBoolean("-allphone_ci", true)
            .getRecognizer();
    // Phonetic search
    File phoneticModel = new File(modelsDir, "phone/fr-phone.lm.dmp");
    recognizer.addAllphoneSearch(PHONE_SEARCH, phoneticModel);

I doubt it will work well on Glass though, it doesn't seem to be very powerful device.

Upvotes: 3

Related Questions