CODEWITHSUNDEEP
google-cloud-vertex-aidialogflow-cxvertex-ai-search
Jin Huynh
Jin Huynh

Reputation: 31

Vertex AI Agent not returning toolCall in response

I am trying to use Function Tools in Agent Builder. According to the Vertex AI documentation here, I should get back toolCall in the response when the agent thinks that I am calling a tool.

{
  "queryResult": {
    "text": "what is the weather in Mountain View",
    "languageCode": "en",
    "responseMessages": [
      {
        "source": "VIRTUAL_AGENT",
        "toolCall": {
          "tool": "<tool-resource-name>",
          "action": "get-weather-tool",
          "inputParameters": {
            "location": "Mountain View"
          }
        }
      }
    ]
  }}

I have tried this the following way:In agent builder playbook, I type in how is the weather. I can see in the network request from dialogflow.clients6.google.com/v3alpha1/ that it contains toolCall and tells me which function is called.The response with playbook:

  "queryResult": {
    "text": "how is the weather in Stockholm?",
    "languageCode": "en",
    "responseMessages": [
      {
        "source": "VIRTUAL_AGENT",
        "toolCall": {
          "tool": "projects/geoconxaiservices/locations/eu/agents/2e495876-b080-4755-9f98-9962ab0cc873/tools/a11594cb-50f5-42cf-9eba-06728ee91e92",
          "action": "getWeather",
          "inputParameters": {
            "location": "Stockholm, Sweden",
            "unit": "celsius"
          }
        }
      }
    ],

When I use the integration with Dialogflow Messenger, it calls dialogflow.googleapis.com/v3/ and I don't get the toolCall in the response.

  "queryResult": {
    "text": "how is the weather in Stockholm?",
    "languageCode": "en",
    "responseMessages": [
      {}
    ],

But it will also fail on following queryInputs since it is waiting for the output from my client side code execution. I get "message": "com.google.apps.framework.request.BadRequestException: Session is waiting for tool call result of tool, and action, getWeather.". I have tested the same flow in both playbook and df messenger (with the same agent) and it doesn't seem that they work the same way. Does anyone know why there is a difference in behavior and why isn't toolCall returned to df messenger?

Upvotes: 3

Views: 486

Answers (2)

Dan Bush
Dan Bush

Reputation: 1

It works in the v3 beta version of the Java client.

import com.google.cloud.dialogflow.cx.v3beta1.*;

Upvotes: 0

Dev Bitcot
Dev Bitcot

Reputation: 1

first of all this is a demo function which you're trying to use. this function has no server. if you want to use the function tool then first deploy your api in any server and put your API's YAML schema in openapi tool.

Upvotes: 0

Related Questions