CODEWITHSUNDEEP
ansibleboto3jmespath
Yehonatan Yochpaz
Yehonatan Yochpaz

Reputation: 73

JMESPathTypeError when using json_query filter in Ansible with starts_with

I am trying to filter results that arrived from boto3 in Ansible.

When I use json query on the results without the "[?starts_with(...)]" it works well, but when adding the starts_with syntax:

"state_machines[?starts_with(name,'hello')].state_machine_arn"

In order to filter results:

{u'boto3': u'1.4.4', u'state_machines': 
[{u'state_machine_arn': u'<state machine arn 1>', u'name': u'hello_world_sfn', u'creation_date': u'2017-05-16 14:26:39.088000+00:00'}, 
{u'state_machine_arn': u'<state machine arn 2>', u'name': u'my_private_sfn', u'creation_date': u'2017-06-08 07:25:49.931000+00:00'}, 
{u'state_machine_arn': u'<state machine arn 3>', u'name': u'alex_sfn', u'creation_date': u'2017-06-14 08:35:07.123000+00:00'}], 
u'changed': True}" }

I expect to get the first state_machine_arn value: "state machine arn 1"

But instead, I get the exception:

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: JMESPathTypeError: In function contains(), invalid type for value: <lamdba_name>, expected one of: ['array', 'string'], received: "unknown" fatal: [localhost]: FAILED!
=> {"failed": true, "msg": "Unexpected failure during module execution.", "stdout": ""}

What can be the problem?

Upvotes: 7

Views: 9800

Answers (2)

200_success
200_success

Reputation: 7582

As an alternative to writing | to_json | from_json | json_query(…) everywhere, you can monkey-patch Ansible's json_query filter by creating the following filter_plugins/json_bug_workaround.py file:

import json
from ansible.parsing.ajson import AnsibleJSONEncoder
from ansible.plugins.filter.json_query import json_query 

class FilterModule(object):
    def filters(self):
        return {
            # Workaround for Unicode bug https://stackoverflow.com/a/44547305
            'json_query': lambda data, query: json_query(
                json.loads(json.dumps(data, cls=AnsibleJSONEncoder)),
                query
            ),
        }

Then you can just use | json_query(…) naturally. This shim does the equivalent of calling | to_json | from_json for you.

You can put it inside your role (roles/role_name/filter_plugins/json_bug_workaround.py) or anywhere in Ansible's plugin search path.

Upvotes: 0

falconizer
falconizer

Reputation: 418

The problem is that json_query filter expects to get a dictionary with ascii strings, but what you're providing it are unicode strings (notice the u'blabla' in your input).

This is an issue with json_query that apparently got introduced in Ansible 2.2.1 (although that is not really clear), here are some more details: https://github.com/ansible/ansible/issues/20379#issuecomment-284034650

I hope this gets fixed in a future version, but for now this is a workaround that worked for us:

"{{ results | to_json | from_json | json_query(jmespath_query) }}"

Where jmespath_query is a variable that contains a starts_with query. This trick of going to and from json turns the unicode strings into ASCII ones :)

Upvotes: 12

Related Questions