CODEWITHSUNDEEP
pythonspeech-recognitiongrammarcmusphinx
user3303504
user3303504

Reputation: 555

Runtime error when using grammars with the SpeechRecognition package

So I've tried using grammars with speech_recognition.recognize_sphinx(), however, I get the following error:

RuntimeError: Decoder_set_fsg returned -1

Here's my code:

Main.py

# Dependencies:
import speech_recognition as sr

# Collect audio sample
r = sr.Recognizer()
print('Please say "perquisition":')
with sr.Microphone() as source:
    audio_en = r.listen(source)
print('Processing...')

# Attempt to convert the speech to text
try:
    print(r.recognize_sphinx(audio_en, grammar='perquisition.gram'))
except sr.UnknownValueError:
    print("Sphinx could not understand audio")
except sr.RequestError as e:
    print("Sphinx error; {0}".format(e))

perquisition.gram

#JSGF V1.0;
grammar perquisition;
// Grammar rule names should be [a-zA-Z0-9] only!
public <perquisition> = ( perquisition );

Any ideas on what's going on?

Upvotes: 1

Views: 602

Answers (1)

user3303504
user3303504

Reputation: 555

There are a few things going on here that are masking the underlying bug. speech_recognition.recognize_sphinx() is just a wrapper for a few CMUsphinx commands, which can be found here at line 746. There's a bit of clutter for this exact problem, so we'll focus on the snippet of code below instead:

# Dependencies
import speech_recognition as sr
import os
import pocketsphinx as ps

# Manually point to the grammar file
grammar = 'search.gram'
try:
    # Point to the model files
    language_directory = os.path.join(os.path.dirname(os.path.realpath(__file__)), "pocketsphinx-data", "en-US")
    acoustic_parameters_directory = os.path.join(language_directory, "acoustic-model")
    language_model_file = os.path.join(language_directory, "language-model.lm.bin")
    phoneme_dictionary_file = os.path.join(language_directory, "pronounciation-dictionary.dict")

    # Create a decoder object with our custom parameters
    config = ps.Decoder.default_config()
    config.set_string("-hmm",
                      acoustic_parameters_directory)  # set the path of the hidden Markov model (HMM) parameter files
    config.set_string("-lm", language_model_file)
    config.set_string("-dict", phoneme_dictionary_file)
    config.set_string("-logfn", os.devnull)  # <--- Prevents you from seeing the actual bug!!!
    decoder = ps.Decoder(config)

    # Convert grammar
    grammar_path = os.path.abspath(os.path.dirname(grammar))
    grammar_name = os.path.splitext(os.path.basename(grammar))[0]
    fsg_path = "{0}/{1}.fsg".format(grammar_path, grammar_name)
    if not os.path.exists(fsg_path):  # create FSG grammar if not available
        jsgf = ps.Jsgf(grammar)
        rule = jsgf.get_rule("{0}.{0}".format(grammar_name))
        fsg = jsgf.build_fsg(rule, decoder.get_logmath(), 7.5)
        fsg.writefile(fsg_path)
        print('Successful JSFG to FSG conversion!!!')

    # Pass the fsg file into the decoder
    decoder.set_fsg(grammar_name, fsg)  # <--- BUG IS HERE!!!

except Exception as e:
    print('Ach no! {0}'.format(e))
finally:
    os.remove('search.fsg')  # Remove again to help prove that the grammar to fsg conversion isn't at fault

Running that code we find the line that the bug pops its head up, but also that the logging information has been turned off! With it on, a lot of text gets dumped into the terminal, which can be a nuisance. In this case, it'd be more useful to turn them back on to discover...

...
ERROR: "fsg_search.c", line 141: The word 'perquisition' is missing in the dictionary
...

Now we're getting somewhere. This leaves us with one of two options. Firstly we can scan the dictionary (pocketsphinx.get_model_path()+'/cmudict-en-us.dict' or similar) to determine if a word is present or not. Then we can decide whether to simply ignore the word, or to add it to the dictionary.

Adding to the dictionary isn't necessarily straight forward... Depending on how similar it is to other words in the dictionary you might be able to get away with it. Otherwise, you'll have to retrain the model as well. A far better explanation on how to do this can be found here. Enjoy.

Upvotes: 1

Related Questions