CODEWITHSUNDEEP
jsonansibleuri
mephisto
mephisto

Reputation: 43

ansible.builtin.uri module - Read and format JSON file content as a string to use in payload

I am trying to use the ansible.builtin.lookup plugin in order to read a JSON file from the local directory and then pass it as the payload to the ansible.builtin.uri module to send a POST message to a URI endpoint.

Following are the contents of my JSON file (config.json):

    {
      "Configuration": {
        "Components": [
          {
            "Name": "A",
            "Attributes": [
              {
                "Name": "A1",
                "Value": "1",
                "Set On Import": "True",
                "Comment": "Read and Write"
              },
              {
                "Name": "A2",
                "Value": "2",
                "Set On Import": "True",
                "Comment": "Read and Write"
              }
            ]
          }
        ]
      }
    }

I need to send the above JSON content as the below string in the payload to ansible.builtin.uri module:

"{\"Configuration\": {\"Components\": [{\"Name\": \"A\", \"Attributes\": [{\"Name\": \"A1\", \"Value\": \"1\", \"Set On Import\": \"True\", \"Comment\": \"Read and Write\"}, {\"Name\": \"A2\", \"Value\": \"2\", \"Set On Import\": \"True\", \"Comment\": \"Read and Write\"}]}]}}"

I am trying to use the lookup plugin with the to_json filter to read and format the JSON content. Following is my playbook:

- name: import scp
  ansible.builtin.uri:
    url: "https://{{ inventory_hostname }}/api/config/actions/import"
    user: "{{ user }}"
    password: "{{ password }}"
    method: POST
    headers:
      Accept: "application/json"
      Content-Type: "application/json"
    body:
      Parameters:
        Type: "LOCAL_FILE"
        Target: "ALL"
        IgnoreCertificateWarning: "Enabled"
      Buffer: "{{ lookup('file', 'config.json') | to_json }}"
    body_format: json
    status_code: 202
    validate_certs: no
    force_basic_auth: yes

However, the uri module double escapes all the new-line and tab characters. Following is the how the payload is sent when I run the playbook:

    "invocation": {
        "module_args": {
            "attributes": null,
            "body": {
                "Buffer": "\"{\\n\\t\\\"Configuration\\\": {\\n\\t\\t\\\"Components\\\": [\\n\\t\\t\\t{\\n\\t\\t\\t\\t\\\"Name\\\": \\\"A\\\",\\n\\t\\t\\t\\t\\\"Attributes\\\": [\\n\\t\\t\\t\\t\\t{\\n\\t\\t\\t\\t\\t\\t\\\"Name\\\": \\\"A1\\\",\\n\\t\\t\\t\\t\\t\\t\\\"Value\\\": \\\"1\\\",\\n\\t\\t\\t\\t\\t\\t\\\"Set On Import\\\": \\\"True\\\",\\n\\t\\t\\t\\t\\t\\t\\\"Comment\\\": \\\"Read and Write\\\"\\n\\t\\t\\t\\t\\t},\\n\\t\\t\\t\\t\\t{\\n\\t\\t\\t\\t\\t\\t\\\"Name\\\": \\\"A2\\\",\\n\\t\\t\\t\\t\\t\\t\\\"Value\\\": \\\"2\\\",\\n\\t\\t\\t\\t\\t\\t\\\"Set On Import\\\": \\\"True\\\",\\n\\t\\t\\t\\t\\t\\t\\\"Comment\\\": \\\"Read and Write\\\"\\n\\t\\t\\t\\t\\t}\\n\\t\\t\\t\\t]\\n\\t\\t\\t}\\n\\t\\t]\\n\\t}\\n}\"",
                "Parameters": {
                    "IgnoreCertificateWarning": "Enabled",
                    "Type": "LOCAL_FILE",
                    "Target": "ALL"
                },
            },
            "body_format": "json",
...
    },

Could you please let me know how I can format the payload with uri module? Appreciate any help.

Edited (5/11/2021): I made the changes as suggested by @mdaniel in his response and used string filter instead of to_json. With the suggested change, I can see the JSON being formatted properly into a string with newline ('\n') and tab ('\t') characters. I tried to use the replace filter to remove the \n and \t characters. However, now the whole string is converted back into the JSON.

Following is the playbook and the output when using the string filter alone:

...
        body:
          Parameters:
            Type: "LOCAL_FILE"
            Target: "ALL"
            IgnoreCertificateWarning: "Enabled"
          Buffer: "{{ lookup('file', 'config.json') | string }}"

$ ansible-playbook import_file.yml -i hosts --tags

...
            "body": {
                "HostPowerState": "On",
                "Buffer": "{\n\t\"Configuration\": {\n\t\t\"Components\": [\n\t\t\t{\n\t\t\t\t\"Name\": \"A\",\n\t\t\t\t\"Attributes\": [\n\t\t\t\t\t{\n\t\t\t\t\t\t\"Name\": \"A1\",\n\t\t\t\t\t\t\"Value\": \"1\",\n\t\t\t\t\t\t\"Set On Import\": \"True\",\n\t\t\t\t\t\t\"Comment\": \"Read and Write\"\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\t\"Name\": \"A2\",\n\t\t\t\t\t\t\"Value\": \"2\",\n\t\t\t\t\t\t\"Set On Import\": \"True\",\n\t\t\t\t\t\t\"Comment\": \"Read and Write\"\n\t\t\t\t\t}\n\t\t\t\t]\n\t\t\t}\n\t\t]\n\t}\n}",
                "Parameters": {
                    "IgnoreCertificateWarning": "Enabled",
                    "Type": "LOCAL_FILE",
                    "Target": "ALL"
                },
            },

Following is the playbook and the output when using replace filter in conjunction with string filter:

...
        body:
          Parameters:
            Type: "LOCAL_FILE"
            Target: "ALL"
            IgnoreCertificateWarning: "Enabled"
          Buffer: "{{ lookup('file', 'config.json') | string | replace('\n', '') | replace('\t', '') }}"
...


$ ansible-playbook import_file.yml -i hosts --tags

...
            "body": {
                "Buffer":  {
                    "Configuration": {
                        "Components": [
                            {
                                "Attributes": [
                                    {
                                        "Comment": "Read and Write",
                                        "Name": "A1",
                                        "Set On Import": "True",
                                        "Value": "1"
                                    },
                                    {
                                        "Comment": "Read and Write",
                                        "Name": "A2",
                                        "Set On Import": "True",
                                        "Value": "2"
                                    }
                                ],
                                "Name": "A"
                            }
                        ]
                    }
                },
                "Parameters": {
                    "IgnoreCertificateWarning": "Enabled",
                    "Type": "LOCAL_FILE",
                    "Target": "ALL"
                },
            },
...

Any pointers on how I remove the \n and \t characters from the string?

Upvotes: 0

Views: 4219

Answers (1)

mdaniel
mdaniel

Reputation: 33203

You have used to_json on a dict value that is, itself, going to be to_json-ed; ansible cannot transmit a python dict over HTTP, so any yaml structure that is not already a string needs to be converted into one first

What you'll want is just that lookup result (which will return a str, not a dict) and then ansible will apply to_json to the whole body: value for the aforementioned reason

However, because ansible is trying to be "helpful", it will auto-coerce a yaml value that it finds starting with { back into a dict -- that's why you just need to send the result of lookup through the | string filter to reinforce to ansible that yes, you really do want it to remain a str in that context

...

    body:
      Parameters:
        Type: "LOCAL_FILE"
        Target: "ALL"
        IgnoreCertificateWarning: "Enabled"
      Buffer: "{{ lookup('file', 'config.json') | string }}"

updated answer approach

In light of the comment discussion that the dict coercion was continuing to be a problem, and the leading space concerned the OP, the alternative approach is to build up the actual payload structure completely, and only "JSON-ify" it before transmission, to keep ansible and jinja on the same page about the data types:

- name: import scp
  vars:
    body_dict:
      Parameters:
        Type: "LOCAL_FILE"
        Target: "ALL"
        IgnoreCertificateWarning: "Enabled"
      # this will be filled in before submission
      # Buffer:
    whitespace_free_config_json: >-
          {{ lookup('file', 'config.json') 
          | regex_replace('[\t\n]', '')
          | string
          }}
  ansible.builtin.uri:
    ...
    body: >-
      {{ body_dict
      | combine({"Buffer": whitespace_free_config_json})
      | to_json }}
    body_format: json
    status_code: 202

Upvotes: 1

Related Questions