Speech recognition using a real time stream

Question

Firstly, in order to clarify my goal: I am using the CSCore library and capturing background audio using the WasapiLoopbackCapture class, and I intend to use that as a real time input for a System.Speech.Recognition recognition engine. That class either outputs the data to a .WAV file or to a Stream. I then tried doing this:

    private void startButton_Click(object sender, EventArgs e)
    {
        _recognitionEngine.UnloadAllGrammars();
        _recognitionEngine.LoadGrammar(new DictationGrammar());

        LoadTargetDevice();
        StartStreamCapture(); // Here I am starting the capture to _stream (MemoryStream type)

        _stream.Position = 0; // Without setting this, I get a stream format exception.

        _recognitionEngine.SetInputToWaveStream(_stream);
        _recognitionEngine.RecognizeAsync(RecognizeMode.Multiple);
    }

The result is that I don't get an exception but I also don't get the SpeechRecognized or SpeechDetected events firing. I suspect this is because the System.Speech.Recognition assembly does not support real time streams. I searched online and someone reports implementing a custom Stream type as a workaround, but I was unable to follow the instructions on the post which were unclear (see Dexter Morgan's reply here).

I am aware this problem is best solved by using a different library or an alternate approach, but I would like to know how to do this makeshift implementation specifically, mostly for knowledge purposes.

Thanks!

@Webruster From communicating with the OP, it seems as though the format they were using is unsupported. See [this image](https://media.discordapp.net/attachments/243146188264701952/406422434947006465/unknown.png) and [this one](https://i.imgur.com/6ZJva7R.png) — JustCarty, Feb 01 '18 at 08:04
@JustCarty so what are you expecting? and what is your point of question than the OP question — The Blue Shirt Developer, Feb 01 '18 at 08:50
@Webruster The OP created a program that listened to real-time audio and would convert to text. The program was not crashing but would not retrieve the audio or firing the subsequent events. The OP wanted to know why that was happening. — JustCarty, Feb 01 '18 at 08:52
Possible duplicate of [Streaming input to System.Speech.Recognition.SpeechRecognitionEngine](https://stackoverflow.com/questions/1682902/streaming-input-to-system-speech-recognition-speechrecognitionengine) — Manuel Allenspach, Feb 07 '18 at 15:20

score 5 · Answer 1 · answered Feb 01 '18 at 10:00

@Justcarty thanks for the clarification, here is my Explanation why Code of OP wont work and what need to be done in order to make it work.

In C# for the speech recongintion and synthesis , you probably confused by the documentation where we are having two Speech DLL's
1. Microsoft Speech DLL (Microsoft.speech.dll) 2. System Speech DLL (System.Speech.Dll)

System.speech dll is a part of the windows OS . The two libraries are similar in the sense that the APIs are almost, but not quite, the same. So, if you’re searching online for speech examples , from the code snippets you get you may not tell whether they explaining to System.Speech or Microsoft.Speech.

So for Adding a Speech to the C# application you need to use the Microsoft.Speech library, not the System.Speech library.

Some of the key differences are summarized belows

|-------------------------|---------------------|
|  Microsoft.Speech.dll    | System.Speech.dll  |
|-------------------------|---------------------|
|Must install separately  |                     |
|                         | Part of the OS      |
|                         |  (Windows Vista+)   |
|-------------------------|---------------------|
|Must construct Grammars  | Uses Grammars or    |
|                           free dictation      |
| ------------------------|--------------------|

For more Read the Following Article , it explains the correct way to implement

Is there no way to do so without the using another DLL? – JustCarty Feb 01 '18 at 14:04 — JustCarty, Feb 01 '18 at 14:04

Speech recognition using a real time stream

1 Answers1