How to send carousel through API.AI?

Question

My bot wants to send a carousel to Google Assistant through API.AI. My understanding is, I need to enclose it inside data -> google, such as:

{
    "data": {
        "google": {
            "expectUserResponse": true,
            "isSsml": false,
            "expectedInputs": [
                {
                    "inputPrompt": {
                        "richInitialPrompt": {
                            "items": [
                                {
                                    "simpleResponse": {
                                        "textToSpeech": "Hello World"
                                    }
                                }
                            ]
                        }
                    },
                    "possibleIntents": [
                        {
                            "intent": "actions.intent.OPTION",
                            "inputValueData": {
                                "@type": "type.googleapis.com/google.actions.v2.OptionValueSpec",
                                "carouselSelect": {
                                    "items": [
                                        {
                                          "optionInfo": {"key": "FOO", "synonyms": ["foo"]},
                                          "title": "Foo",
                                          "image": {"url": "http://example.com/", "accessibilityText": "Foo"}
                                        },
                                        {
                                            "optionInfo": {"key": "BAR", "synonyms": ["bar"]},
                                            "title": "Bar",
                                            "image": {"url": "http://example.com/", "accessibilityText": "Bar"}
                                        }
                                    ]
                                }
                            }
                        }
                    ]
                }
            ]
        }
    }
}

But it doesn't work. What is the proper format?

Answer 1

If you are testing this through the Simulator, there should have been a validation error that appeared that would give you at least a little guidance about what is missing. If you didn't even get that, there may be a problem with the other parts besides the data.google object such that api.ai had problems with it.

There are a number of things that, at a glance, could be the problem. You can't just stick a conversation webhook response in the api.ai response. See https://developers.google.com/actions/apiai/webhook#response for the documentation, but here are a few things that I see that could be issues

• The expectedInputs property shouldn't be there.
• Your data.google.expectedInputs.possibleIntents property should be at data.google.systemIntent
• You still need to provide the api.ai fields, such as a basic speech property
• The data.google.expectedInputs.inputPrompt.richInitialPrompt would be at data.google.richResponse

Here is some JSON that works for me:

{
    "speech": "Hello",
    "contextOut": [
        {
            "name": "_actions_on_google_",
            "lifespan": 100,
            "parameters": {}
        }
    ],
    "data": {
        "google": {
            "expectUserResponse": true,
            "richResponse": {
                "items": [
                    {
                        "simpleResponse": {
                            "textToSpeech": "Hello"
                        }
                    }
                ],
                "suggestions": []
            },
            "systemIntent": {
                "intent": "actions.intent.OPTION",
                "data": {
                    "@type": "type.googleapis.com/google.actions.v2.OptionValueSpec",
                    "carouselSelect": {
                        "items": [
                            {
                                "title": "Foo",
                                "image": {
                                    "url": "http://example.com/foo.jpg",
                                    "accessibilityText": "Foo title"
                                },
                                "optionInfo": {
                                    "key": "foo-key",
                                    "synonyms": [
                                        "foo-alt-1",
                                        "foo-alt-2"
                                    ]
                                }
                            },
                            {
                                "title": "Bar",
                                "image": {
                                    "url": "http://example.com/bar.jpg",
                                    "accessibilityText": "Bar title"
                                },
                                "optionInfo": {
                                    "key": "bar-key",
                                    "synonyms": [
                                        "bar-alt-1",
                                        "bar-alt-2"
                                    ]
                                }
                            }
                        ]
                    }
                }
            }
        }
    }
}