CODEWITHSUNDEEP
bashcurl
Paul Cezanne
Paul Cezanne

Reputation: 8741

bash, braces and quotes and curl -d

I'm trying to send some headers and key value pairs of data to my server using curl. It looks like I'm getting a bash quoting problem, my key value pairs are being treated like hosts.

curl -v --header "Content-Type: application/json" \
--header "Foo: Bar 123456"  \
-d \"\{ "baz": "glern", "froboz": "foo again"\}\" \
https://example.com > foo.html 2> error.txt

example.com is telling me:

HTTP 404 Not Found: URL or Document not found (dns_unresolved_hostname) 

HTTP 404: Your requested host "baz" could not be resolved by DNS. The document at the specified URL does not exist.

HTTP 404 Not Found: URL or Document not found (dns_unresolved_hostname) 

HTTP 404: Your requested host "glern," could not be resolved by DNS. The document at the specified URL does not exist.

and error.txt starts off like this:

* Rebuilt URL to: baz:/
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 10.196.133.236...
* TCP_NODELAY set
* Connected to http.proxy.mycompanyproxy.com (10.196.133.236) port 8000 (#0)
> POST http://baz:/ HTTP/1.1
> Host: baz
> User-Agent: curl/7.54.0
> Accept: */*
> Proxy-Connection: Keep-Alive
> Content-Type: application/json
> Foo: Bar 123456
> Content-Length: 2
> } [2 bytes data]
* upload completely sent off: 2 out of 2 bytes
< HTTP/1.1 404 Not Found
...

I certainly don't mean to hit the hosts baz, glern, etc... and it certainly looks like a shell quoting problem. But after trying many different things, like adding extra \" near the existing bare " around my key value pairs, I'm a bit lost here.

Upvotes: 1

Views: 1266

Answers (2)

chepner
chepner

Reputation: 531205

Consider what your data would look like if you simply had a file:

$ cat tmp.json
{ "baz": "glern", "froboz": "foo again" }

In that case, you'd simply use

$ curl ... -d "$(cat tmp.json)"

Now just do the command substitution "manually", so to speak (switching from double to single quotes to avoid escaping a bunch of quotes on the command line):

$ curl ... -d '{ "baz": "glern", "froboz": "foo again" }'
# with double quotes, it's
#  curl ... -d "{ \"baz\": \"glern\", \"froboz\": \"foo again\" }"
# ... ew.

In general, though, it's better to let a tool like jq generate JSON for you instead of trying to write it manually; this can simplify quoting greatly when it is needed.

$ curl ... -d "$(jq -n '{baz: "glern", froboz: "foo again"}')"

It's almost mandatory to use such a tool if your JSON isn't hard-coded, but from variables like {"baz": "$SOME_VAR"}. Instead of jumping through hoops to make sure everything in SOME_VAR is properly escaped, let jq do it for you: jq -n --argjson x "$SOME_VAR" '{baz: $x}'. Note that $x is not a shell variable; it's a jq variable.

Upvotes: 3

Jonathan Leffler
Jonathan Leffler

Reputation: 753970

After the -d, only the \"\{ forms an argument to -d. It isn't absolutely clear what you're after, but maybe:

curl -v --header "Content-Type: application/json" \
     --header "Foo: Bar 123456"  \
     -d '{ "baz": "glern", "froboz": "foo again"}' \
     https://example.com > foo.html 2> error.txt

That way, the argument to -d is:

{ "baz": "glern", "froboz": "foo again"}

There's no call to use double quotes around a string that contains copious double quotes; that just makes life harder.

Upvotes: 3

Related Questions