CODEWITHSUNDEEP
arraysjsonbashcurlwindows-subsystem-for-linux
Invisible999
Invisible999

Reputation: 577

Bash: works at command line but getting 'curl: (1) Protocol ""https" not supported or disabled in libcurl' when using in a script

Got stuck with the situation that a curl request works fine with the command line but does not work at a script giving weird errors. Here is the part of the script:

#we already got the $token
#vars defining curl parameters
dc="NA EU AU SEA JP"
curl_url='https://example.com/api/config/v1/'
curl_request='alertingProfiles?'
curl_properties='tenant=all&stage=prd&cluster='
curl_auth='"Authorization: Bearer '"$token"'"'
for k in $dc; do

        # Download the json for each DC using the curl request
        curl_combined=\""$curl_url$curl_request$curl_properties$k"\"; echo "$curl_combined"
        curlCMD=( -vv --location --request GET "$curl_combined"  -H "$curl_auth" -o \""$k.json"\")
        echo "${curlCMD[@]}"
        curl "${curlCMD[@]}"
        done

When I run this in a script I see the following output: First, the output from the first echo command

"https://example.com/api/config/v1/alertingProfiles?tenant=all&stage=prd&cluster=NA"

Then, the output from the second echo command which is what curl should take as a parameter, I've skipped the bearer token, but trust me, it is not a problem here.

-vv --location --request GET "https://example.com/api/config/v1/alertingProfiles?tenant=all&stage=prd&cluster=NA" -H "Authorization: Bearer " -o "NA.json"

And I am getting following output

Note: Unnecessary use of -X or --request, GET is already inferred.
* Protocol ""https" not supported or disabled in libcurl
* Closing connection -1
curl: (1) Protocol ""https" not supported or disabled in libcurl

Please note two double quotes in the message above in front of https.

If I simply copy/paste the above second echo output in the command line right after curl and execute, it works as expected and downloads JSON in the file.

If I remove " from curl_combined var, the request is sent without double quotes and now I am getting the message curl: (56) Unexpected EOF from the script.

If I copy/paste the second echo output (without double quotes), supply it to curl, and execute from the command line, I am getting 403 message until I put double or single quotes around the https request.

In short - it seems that double quotes around the https request is necessary but for some reason, curl does not take it.

So what went wrong here? I don't mind even to write the constructed URL in a file and read from it or use herestring if it helps, but want to know what exactly went wrong here.

Thanks.

P.S. Removing --location or --request GET or both does not change the outcome.

Upvotes: 0

Views: 1096

Answers (2)

Invisible999
Invisible999

Reputation: 577

This solution works replacing curl request with array to the following

curl -vv \
--location \
--request GET \
"$curl_combined" \
--header "Authorization: Bearer ${token}" \
-o "$k.json"

But it is ugly, let's say politely...

Upvotes: 0

dash-o
dash-o

Reputation: 14452

The script is using passing the "${curlCMD[@]}" array as parameter to curl. This approach will handle the correct expansion of the url, regardless of any special characters in it. There is no need to wrap the curl_combined with additional quotes using the \" construct.

Suggesting replacing setting curl_combined with:

curl_combined="$curl_url$curl_request$curl_properties$k"
echo "$curl_combined"

Upvotes: 1

Related Questions