Google Action / Dialogflow : how to ask for geolocation

Question

I'm trying to implement a simple app for Google Assistant. All works fine, but now I have a problem with the "permission" helper : https://developers.google.com/actions/assistant/helpers#helper_intents

I have an intent connected with webhook to my java application. When an user types a sentence similar to "near to me", I want to ask to him his location and then use lat/lon to perform a search.

es: Brazilian restaurant near to me

1. my intent "searchRestaurant" is fired

2. I receive the webhook request and I parse it

3. if I find a parameter that is connected to a sentence like "near to me", so instead to response with a "Card" or a "List" I return a JSON that represent the helper request :

---

{
"conversationToken": "[]",
"expectUserResponse": true,
"expectedInputs": [
    {
        "inputPrompt": {
            "initialPrompts": [
                {
                    "textToSpeech": "PLACEHOLDER_FOR_PERMISSION"
                }
            ],
            "noInputPrompts": []
        },
        "possibileIntents": [
            {
                "intent": "actions.intent.PERMISSION",
                "inputValueData": {
                    "@type": "type.googleapis.com/google.actions.v2.PermissionValueSpec",
                    "optContext": "Posso accedere alla tua posizione?",
                    "permission": [
                        "NAME",
                        "DEVICE_PRECISE_LOCATION"
                    ]
                }
            }
        ]
    }
]
}

but something seems to be wrong, and I receive an error:

"{\n \"responseMetadata\": {\n \"status\": {\n \"code\": 10,\n \"message\": \"Failed to parse Dialogflow response into AppResponse because of empty speech response\",\n \"details\": [{\n \"@type\": \"type.googleapis.com/google.protobuf.Value\",\n \"value\": \"{\\"id\\":\\"1cc45c5e-c398-4ea7-98a5-408f31ce142d\\",\\"timestamp\\":\\"2018-08-02T14:45:05.752Z\\",\\"lang\\":\\"it\\",\\"result\\":{},\\"alternateResult\\":{},\\"status\\":{\\"code\\":206,\\"errorType\\":\\"partial_content\\",\\"errorDetails\\":\\"Webhook call failed. Error: Failed to parse webhook JSON response: Cannot find field: conversationToken in message google.cloud.dialogflow.v2.WebhookResponse.\\"},\\"sessionId\\":\\"1533221100163\\"}\"\n }]\n }\n }\n}"

The "conversationToken" is filled, so I don't understand the error message. Maybe I'm trying to perform the operation in a wrong way.

So, which is the correct way to call this helper?

--> I've created a second intent "askGeolocation" that have "actions_intent_PERMISSION" as "Event", and ... if I understand correctly the documentation, should be trigger if the request for helper is correct.

How can I get this working?

UPDATE :

I find some example of the json response for ask permission and seems that it should be different from the one above that i'm using :

https://github.com/dialogflow/fulfillment-webhook-json/blob/master/responses/v2/ActionsOnGoogle/AskForPermission.json

{
"payload": {
"google": {
  "expectUserResponse": true,
  "systemIntent": {
    "intent": "actions.intent.PERMISSION",
    "data": {
      "@type": "type.googleapis.com/google.actions.v2.PermissionValueSpec",
      "optContext": "To deliver your order",
      "permissions": [
        "NAME",
        "DEVICE_PRECISE_LOCATION"
      ]
        }
      }
    }
  }
}

so, i've implemented it and now the response seems to be good (no more error on parsing it), but i still receive an error on it validation :

UnparseableJsonResponse API Version 2: Failed to parse JSON response string with 'INVALID_ARGUMENT' error: "permission: Cannot find field."

so, a problem still persist. Anyone know the cause?

Thanks

Answer 1

There are samples on this topic specifically that guide you through exactly what's needed for requesting permissions in Node and Java.

Note: There are helper intents samples available in Node and Java as well.

Answer 2

After some tests i found the problem. I was returning a wrong json repsonse with "permission" instead of "permissions":

"permission**s**": [
    "NAME",
    "DEVICE_PRECISE_LOCATION"
]

So the steps to ask for location are correct. I report them here as a little tutorial in order to help who is facing on it for the first time:

1) In DialogFlow, add some logic to your Intent, in order to understand when is ok to ask to user his location. In my case, i've added a "parameter" that identify sentences like "nearby" and so on.

2) When my Intent is fired i receive to my java application a request like this :

...
"queryResult": {
"queryText": "ristorante argentino qui vicino",
"action": "bycategory",
"parameters": {
  "askgeolocation": "qui vicino",
  "TipoRistorante": ["ristorante", "argentino"]
},
...

3) If "askgeolocation" parameter is filled, instead to return a "simple message" o other type of message, i return a json for ask the permission to geolocation :

{
"payload": {
"google": {
  "expectUserResponse": true,
  "systemIntent": {
    "intent": "actions.intent.PERMISSION",
    "data": {
      "@type": "type.googleapis.com/google.actions.v2.PermissionValueSpec",
      "optContext": "To deliver your order",
      "permissions": [
        "NAME",
        "DEVICE_PRECISE_LOCATION"
     ]
        }
      }
    }
  }
}

4) You MUST have a second Intent that is configured with "actions_intent_PERMISSION " event :

• No training phrases
• No Action and params
• No Responses

But with Fulfillment active :

5) Once your response arrive to Google Assistant this is the message that appear :

6) Now, if user answer "ok" you receive this json on your webhook :

     {
   "responseId": "32cf46cf-80d8-xxxxxxxxxxxxxxxxxxxxx",
   "queryResult": {
     "queryText": "actions_intent_PERMISSION",
     "action": "geoposition",
     "parameters": {
     },
 "allRequiredParamsPresent": true,
 "fulfillmentMessages": [{
   "text": {
     "text": [""]
   }
 }],
 "outputContexts": [{
   "name": "projects/xxxxxxxxxxxxxxxxxxxxx"
 }, {
   "name": "projects/xxxxxxxxxxxxxxxxxxxxx",
   "parameters": {
     "PERMISSION": true
   }
 }, {
   "name": "projects/xxxxxxxxxxxxxxxxxxxxx"
 }, {
   "name": "projects/xxxxxxxxxxxxxxxxxxxxx"
 }, {
   "name": "projects/xxxxxxxxxxxxxxxxxxxxx"
 }, {
   "name": "projects/xxxxxxxxxxxxxxxxxxxxx"
 }],
 "intent": {
   "name": "projects/xxxxxxxxxxxxxxxxxxxxx",
   "displayName": "geoposition"
 },
 "intentDetectionConfidence": 1.0,
 "languageCode": "it"
 },
 "originalDetectIntentRequest": {
 "source": "google",
 "version": "2",
 "payload": {
   "isInSandbox": true,
   "surface": {
     "capabilities": [{
       "name": "actions.capability.MEDIA_RESPONSE_AUDIO"
     }, {
       "name": "actions.capability.SCREEN_OUTPUT"
     }, {
       "name": "actions.capability.AUDIO_OUTPUT"
     }, {
       "name": "actions.capability.WEB_BROWSER"
     }]
   },
   "requestType": "SIMULATOR",
   "inputs": [{
     "rawInputs": [{
       "inputType": "KEYBOARD"
     }],
     "arguments": [{
       "textValue": "true",
       "name": "PERMISSION",
       "boolValue": true
     }],
     "intent": "actions.intent.PERMISSION"
   }],
   "user": {
     "lastSeen": "2018-08-03T08:55:20Z",
     "permissions": ["NAME", "DEVICE_PRECISE_LOCATION"],
     "profile": {
       "displayName": ".... full name of the user ...",
       "givenName": "... name ...",
       "familyName": "... surname ..."
     },
     "locale": "it-IT",
     "userId": "xxxxxxxxxxxxxxxxxxxxx"
   },
   "device": {
     "location": {
       "coordinates": {
         "latitude": 45.xxxxxx,
         "longitude": 9.xxxxxx
       }
     }
   },
   "conversation": {
     "conversationId": "xxxxxxxxxxxxxxxxxxxxx",
     "type": "ACTIVE",
     "conversationToken": "[]"
   },
   "availableSurfaces": [{
     "capabilities": [{
       "name": "actions.capability.SCREEN_OUTPUT"
     }, {
       "name": "actions.capability.AUDIO_OUTPUT"
     }, {
       "name": "actions.capability.WEB_BROWSER"
     }]
   }]
     }
   },
   "session": "projects/xxxxxxxxxxxxxxxxxxxxx"
 }

that contains, name/surname and latitude/longitude. This information can be saved in your application, in order to not perform again this steps.

I hope this helps.

Davide

Answer 3

In your intent, you can ask for a parameter with a custom Entity. This you can do like this:

• entity you can define as "near"
• put all the synonyms for near for which you want to trigger location permission in this entity
• do not mark this parameter as "required"
• do not put any prompt
• in the training phrases, add some phrases with this parameter
• in your webhook, keep a check on the parameter, if present ask for permission if not continue.
• add a permission event to another intent
• do your post permission processing in that intent

Entity

Intent


I hope you get it.