11

How would I go about coding a voice trigger to navigate Google Glass Cards?

This is how I see it happening:

1) "Ok Glass, Start My Program"

2) Application begins and shows the first card

3) User can say "Next Card" to move to the next card 
(somewhat the equivalent of swiping forward when in the timeline)

4) User can say "Previous Card" to go back 

The cards that I need to display are simple text and images, I'm wondering if I can setup a listener of some type to listen for voice commands while the card is being shown.


I've researched Glass voice command nearest match from given list but wasn't able to run the code, although I do have all the libraries.

side note: It's important that the user still see the card when using the voice command. Also his hands are busy so tap/swipe isn't an option.

Any ideas on how to control the timeline within my Immersion app using only voice control? would be greatly appreciated!

I am tracking https://code.google.com/p/google-glass-api/issues/detail?id=273 as well.


My ongoing research made me look back at Google Glass Developer to use Google's suggested way of listening to gestures: https://developers.google.com/glass/develop/gdk/input/touch#detecting_gestures_with_a_gesture_detector

How can we activate these gestures with voice commands?


Android just beta-released wearable devices upgrade for android http://developer.android.com/wear/notifications/remote-input.html, Is there a way we can use this to answer my question? it still feels like we are still 1-step away since we can call on the service but not have it "sleep" and "wake up" as a background service when we talk.

Community
  • 1
  • 1
Drace
  • 691
  • 7
  • 22
  • The link to the S-O answer you gave worked perfectly for me and allows exactly the functionality it sounds like you are after. what issue are you having with this? Are you doing this within a cardscrollview? – Ben Feb 25 '14 at 14:17
  • Did you ever figure this out? I am trying to do something very similar – RoboCop87 Mar 02 '14 at 09:28
  • Not yet, I'm hoping the next Glass update will contain the feature - It's really a must if they want people to adopt Glass. We need to have a conversation with Glass. – Drace Mar 04 '14 at 15:16

5 Answers5

4

this thing define in onCreate method

mAudioManager = (AudioManager) context.getSystemService(Context.AUDIO_SERVICE); 
    //  mAudioManager.setStreamSolo(AudioManager.STREAM_VOICE_CALL, true);

    sr = SpeechRecognizer.createSpeechRecognizer(context);       
    sr.setRecognitionListener(new listener(context));   

    //      intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL, "en-US");
    intent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);        
    intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL,RecognizerIntent.LANGUAGE_MODEL_FREE_FORM);
    intent.putExtra(RecognizerIntent.EXTRA_CALLING_PACKAGE,context.getPackageName());
    sr.startListening(intent);
    Log.i("111111","11111111"+"in");

This listener class simply add in your class

class  listener implements RecognitionListener          
{
    Context context1;
    public listener(Context context)
    {
        //Log.i("onError startListening","enter"+"nam");
        context1=context;
    }
    public void onReadyForSpeech(Bundle params)
    {
        //Log.d(TAG, "onReadyForSpeech");
    }
    public void onBeginningOfSpeech()
    {
        //Log.d(TAG, "onBeginningOfSpeech");
    }
    public void onRmsChanged(float rmsdB)
    {
        //Log.d(TAG, "onRmsChanged");
    }
    public void onBufferReceived(byte[] buffer)
    {
        //Log.d(TAG, "onBufferReceived");
    }
    public void onEndOfSpeech()
    {
        //Log.d(TAG, "onEndofSpeech");
        sr.startListening(intent);
    }
    public void onError(int error)
    {
        //Log.d(TAG,  "error " +  error);
        //7 -No recognition result matched.
        //9 - vInsufficient permissions 
        //6 - No speech input 
        //8 RecognitionService busy. 
        //5 Other client side errors. 
        //3 Audio recording error.  
        //  mText.setText("error " + error);

        if(error==6 || error==7 || error==4  || error==1 || error==2 || error==5 || error==3 || error==8 || error==9 )
        { 
            sr.startListening(intent);
            //Log.i("onError startListening","onError startListening"+error);
        }
    }
    public void onResults(Bundle results)                   
    {
        //Log.v(TAG,"onResults" + results);
        ArrayList data = results.getStringArrayList(SpeechRecognizer.RESULTS_RECOGNITION);
        for (int i = 0; i < data.size(); i++)
        {
            //Log.d(TAG, "result " + data.get(i));
            //str += data.get(i);

        //Toast.makeText(context1, "results: "+data.get(0).toString(), Toast.LENGTH_LONG).show();
        //Log.v("my", "output"+"results: "+data.get(0).toString());

        //sr.startListening(intent);
                   }
    }
    public void onPartialResults(Bundle partialResults)
    {
        //Log.d(TAG, "onPartialResults");
    }
    public void onEvent(int eventType, Bundle params)
    {
        //Log.d(TAG, "onEvent " + eventType);
    }
}
pfnuesel
  • 10,855
  • 11
  • 51
  • 63
Nehalkumar
  • 217
  • 2
  • 15
  • Could you please take a look over here http://stackoverflow.com/questions/28828928/activity-has-leaked-serviceconnection-android-speech-speechrecognizerconnection I'm trying to use your piece of code but I can't seem to get it working – NoSixties Mar 03 '15 at 12:38
2

I'm writing out the entire code in detail since it took me such a long time to get this working.. perhaps it'll save someone else valuable time.

This code is the implementation of Google Contextual Voice Commands as described on Google Developers here: Contextual voice commands

ContextualMenuActivity.java

   package com.drace.contextualvoicecommands;

    import android.app.Activity;
    import android.os.Bundle;
    import android.view.Menu;
    import android.view.MenuItem;
    import com.drace.contextualvoicecommands.R;
    import com.google.android.glass.view.WindowUtils;

    public class ContextualMenuActivity extends Activity {

    @Override
    protected void onCreate(Bundle bundle) {
        super.onCreate(bundle);

        // Requests a voice menu on this activity. As for any other
        // window feature, be sure to request this before
        // setContentView() is called
        getWindow().requestFeature(WindowUtils.FEATURE_VOICE_COMMANDS);
        setContentView(R.layout.activity_main);
    }

    @Override
    public boolean onCreatePanelMenu(int featureId, Menu menu) {
        if (featureId == WindowUtils.FEATURE_VOICE_COMMANDS) {
            getMenuInflater().inflate(R.menu.main, menu);
            return true;
        }
        // Pass through to super to setup touch menu.
        return super.onCreatePanelMenu(featureId, menu);
    }

    @Override
    public boolean onCreateOptionsMenu(Menu menu) {
        getMenuInflater().inflate(R.menu.main, menu);
        return true;
    }

    @Override
    public boolean onMenuItemSelected(int featureId, MenuItem item) {
        if (featureId == WindowUtils.FEATURE_VOICE_COMMANDS) {
            switch (item.getItemId()) {
                case R.id.dogs_menu_item:
                    // handle top-level dogs menu item
                    break;
                case R.id.cats_menu_item:
                    // handle top-level cats menu item
                    break;
                case R.id.lab_menu_item:
                    // handle second-level labrador menu item
                    break;
                case R.id.golden_menu_item:
                    // handle second-level golden menu item
                    break;
                case R.id.calico_menu_item:
                    // handle second-level calico menu item
                    break;
                case R.id.cheshire_menu_item:
                    // handle second-level cheshire menu item
                    break;
                default:
                    return true;
            }
            return true;
        }
        // Good practice to pass through to super if not handled
        return super.onMenuItemSelected(featureId, item);
    }
    }

activity_main.xml (layout)

 <?xml version="1.0" encoding="utf-8"?>
    <RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
        xmlns:tools="http://schemas.android.com/tools"
        android:layout_width="match_parent"
        android:layout_height="match_parent" >

          <TextView
        android:id="@+id/coming_soon"
        android:layout_alignParentTop="true"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text="@string/voice_command_test"
        android:textSize="22sp"
        android:layout_marginRight="40px"
        android:layout_marginTop="30px"
        android:layout_marginLeft="210px" /> 
    </RelativeLayout>

strings.xml

<resources>
<string name="app_name">Contextual voice commands</string>
<string name="voice_start_command">Voice commands</string>
<string name="voice_command_test">Say "Okay, Glass"</string>
<string name="show_me_dogs">Dogs</string>
<string name="labrador">labrador</string>
<string name="golden">golden</string>
<string name="show_me_cats">Cats</string>
<string name="cheshire">cheshire</string>
<string name="calico">calico</string>
</resources>

AndroidManifest.xml

 <manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="com.drace.contextualvoicecommands"
    android:versionCode="1"
    android:versionName="1.0" >

    <uses-sdk
        android:minSdkVersion="19"
        android:targetSdkVersion="19" />

    <uses-permission android:name="com.google.android.glass.permission.DEVELOPMENT"/>

    <application
        android:allowBackup="true"
        android:icon="@drawable/ic_launcher"
        android:label="@string/app_name" >

       <activity
            android:name="com.drace.contextualvoicecommands.ContextualMenuActivity"
            android:label="@string/app_name" >
            <intent-filter>
                <action android:name="com.google.android.glass.action.VOICE_TRIGGER" />
            </intent-filter>

            <meta-data
                android:name="com.google.android.glass.VoiceTrigger"
                android:resource="@xml/voice_trigger_start" />
        </activity>

    </application>
    </manifest>

It's been Tested and works great under Google Glass XE22 !

Drace
  • 691
  • 7
  • 22
  • I would say that this isn't the exact awnser to your own question since this still requires you to say "ok glass" in app before you are able to give any other commands. – NoSixties Mar 03 '15 at 07:42
0

You can try the snippet here: https://github.com/pscholl/glass_snippets/tree/master/hotword_detection.

pscholl
  • 502
  • 4
  • 6
0

You may want to try the contextual voice commands available in the GDK. While it does temporarily cover the screen with a menu, it allows voice-only input.

https://developers.google.com/glass/develop/gdk/voice

Lisa Wray
  • 2,204
  • 14
  • 22
  • Yes, however the purpose is to have a surgeon works hands free while looking at what's he's working on, the contextual voice commands needs to run as a background service, I'm attempting a prototype - will get back soon – Drace Jul 19 '14 at 19:25
0

I did something very similar for one of my applications. It doesn't require the ok glass screen at all, but the user does need to know the commands ahead of time. I explained a bit of it and provided links on this question: Check out my answer here: Glass GDk : Contextual voice commands without the "Ok Glass"

I hope this helps!

Community
  • 1
  • 1
RoboCop87
  • 679
  • 1
  • 5
  • 20
  • I will look into it and let you know, thanks! been a long time coming for a solution. – Drace Jul 25 '14 at 22:26
  • I tried running your code but since I didn't root my device, I couldn't install python. Do you happen to have a Voice recognition sample without python? I'm unable to test your code (which looks like it works). thanks! – Drace Oct 14 '14 at 16:57
  • @Loadparts the python code is for a computer that is connect via bluetooth to Glass. You want the code in the android directory. No need to install python https://github.com/RIVeR-Lab/google_glass_driver/tree/master/android/RobotManager/src/com/riverlab/robotmanager – RoboCop87 Oct 14 '14 at 17:09
  • I've got your android app installed.. could you tell me a few sample voice commands that I can run, even if the main screen shows "not connected". and advice on how to test the voice triggers would be greatly appreciated! – Drace Oct 14 '14 at 19:29
  • @Loadparts when it is first run, it listens for commands to connect to a bluetooth device. This isn't going to help you too much. I suggest, if you have the whole code and not just the apk, go into the VoiceRecognitionThread, setup method, under the comments "set up command listener". You will see two references to mConnectCommands. Change those references to mSystemCommands. This will give you access to the commands shown in that same method without needing to connect to a bluetooth device. If you use the view messages or view robots commands, you wont see anything because you... – RoboCop87 Oct 14 '14 at 19:41
  • Arent connected to anything and havent received any messages. If you feel so inclined, you can hard code some examples so they show up. If you did have robots or messages, you could scroll through the cardscrollview using next/previous commands. Sorry that there are no demo programs for non-ROS users, if you had ROS you could run the pre-made demos I made (those are the python files) – RoboCop87 Oct 14 '14 at 19:44
  • 1
    ^and if you didnt mind waiting a day, I could probably put together a out-of-the-box demo that would let you play with it – RoboCop87 Oct 14 '14 at 19:46
  • I am trying to get a working demo so I can use it (and I'm sure the community will benefit) . I am grateful for any help you wish to provide as I am struggling with it. PS: I made the changes of mConnectCommands->mSystemCommands with no success. Also, when looking at Devices, my Iphone 4 shows up (not sure if that helps). but thank you, I will keep working on this but any help is greatly appreciated – Drace Oct 14 '14 at 19:53
  • Hi, have you been able to make that demo? I can't seem to get the voice recognition working to test it out (although I trust it works).. any help is greatly appreciated! – Drace Oct 16 '14 at 16:37
  • Its still sitting on my queue of things to do, but I promise I havent forgotten about it. Im sorry that you are continuing to have trouble testing it! – RoboCop87 Oct 16 '14 at 16:39
  • Thank you for remembering! I'll keep at it meanwhile. – Drace Oct 16 '14 at 16:42
  • Hi Loadparts, sorry for the delay, I've made the demo and pushed it to this repo: https://github.com/nwotero/google_glass_driver_demo/blob/master/android/RobotManager/README.txt While I'm confident it will work, I actually can't test it on my own Glass until next Monday (Because I won't be in possession of it until then). If you continue to have problems with it please e-mail me at notero87@gmail.com and I'll get it sorted out. – RoboCop87 Oct 20 '14 at 03:49
  • 1
    isn't there a way for you to make a library out of this? I bet that would help out a lot of people and it would be easier for everyone to actually use it. Just a thought since I'm trying to get something similair working. – NoSixties Mar 02 '15 at 12:01
  • @NoSixties I agree with you I'm trying something similair as well and I'm having no luck so far with the code provided. –  Mar 02 '15 at 12:12
  • The functionality that this depends on was removed after version 18. I havent been able to find a workaround for newer versions. Sorry if this ate a lot of your time – RoboCop87 Mar 02 '15 at 15:28