CODEWITHSUNDEEP
javascriptnode.jsspeech-recognitionwebrtc
jpen
jpen

Reputation: 2147

Real time speech recognition using WebRTC, Node.js and speech recognition engine

A. What I am trying to implement.

A web application allowing real-time speech recognition inside web browser (like this).

B. Technologies I am currently thinking of using to achieve A.

C. Very basic workflow

  1. Web browser establishes connection to Node server (server acts as a signaling server and also serves static files)
  2. Web browser acquires audio stream using getUserMedia() and sends user's voice to Node server
  3. Node server passes audio stream being received to speech recognition engine for analysis
  4. Speech recognition engine returns result to Node server
  5. Node server sends text result back to initiating web browser
  6. (Node server performs step 1 to 5 to process requests from other browsers)

D. Questions

  1. Would Node.js be suitable to achieve C?
  2. How could I pass received audio streams from my Node server to a speech recognition engine running separately from the server?
  3. Could my speech recognition engine be running as another Node application (if I use Pocketsphinx)? So my Node server communicates to my Node speech recognition server.

Upvotes: 12

Views: 17577

Answers (2)

jesup
jesup

Reputation: 6982

You should contact Andre Natal, who has shown demos similar to this at last fall's Firefox Summit, and is now on a Google Summer of Code project implementing offline speech recognition in Firefox/FxOS: http://cmusphinx.sourceforge.net/2014/04/speech-projects-on-gsoc-2014/

Upvotes: 6

Nikolay Shmyrev
Nikolay Shmyrev

Reputation: 25220

Would Node.js be suitable to achieve C?

Yes, though there are no hard requirements for that. Some people are running servers with gstreamer, for example check

http://kaljurand.github.io/dictate.js/

node should be fine too.

How could I pass received audio streams from my Node server to a speech recognition engine running separately from the server?

There are many ways for node-to-node communication. One of them is http://socket.io. There are also plain sockets. The particular framework depends on your requirements for fault-tolerance and scalability.

Could my speech recognition engine be running as another Node application (if I use Pocketsphinx)? So my Node server communicates to my Node speech recognition server.

Yes, sure. You can create a node module to warp pocketsphinx API.

UPDATE: check this, it should be similar to what you need:

http://github.com/cmusphinx/node-pocketsphinx

Upvotes: 9

Related Questions