Oracle UTL_HTTP request returns a shorter response than when the same request is done in Python or curl

Question

I'm accessing a REST service for data query and download. This is the very first call that performs the authentication. The response is a json structure that contains an authentication token.

When I do this call with curl ...

$ curl -v -X POST ${AUTH_URL} \
   -H 'Content-Type: application/x-www-form-urlencoded' \
   -d 'apikey='${API_KEY}'&grant_type=api_key&client_id=IDP'

... I get the following response.

First the headers:

< server: IIS
< date: Thu, 25 Feb 2021 17:59:34 GMT
< content-type: application/json
< content-length: 1500
< x-content-type-options: nosniff
< x-xss-protection: 1; mode=block
< strict-transport-security: max-age=31536000; includeSubdomains;
< cache-control: no-store
< set-cookie: KC_RESTART=; Version=1; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Max-Age=0; Path=/auth/realms/IDP/; HttpOnly
< pragma: no-cache
< x-frame-options: SAMEORIGIN
< referrer-policy: no-referrer
< vary: Origin
< via: 1.1 google
< alt-svc: clear

And the content:

{"access_token":"eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICIxMThSRENzZTlqUWR4UVRnSkt2ZXlvSHBaaWE4R0pIVEU5RjJPSmE1M3N3In0.eyJleHAiOjE2MTQyOTAzNzQsImlhdCI6MTYxNDI3NTk3NCwianRpIjoiNzBkMmMwZGMtZWY3Yy00NDM5LWJiNmUtNmQ4MDEzZGU2YTU0IiwiaXNzIjoiaHR0cHM6Ly9hdXRoZW50aWNhdGUuZm91bmRhdGlvbi5hcGkub25lYXRsYXMuYWlyYnVzLmNvbS9hdXRoL3JlYWxtcy9JRFAiLCJhdWQiOiJJRFAiLCJzdWIiOiJmNmI0ZjVkNC0zMzZiLTRlMTctODc3Ni1jNjA1ZTczNTRjYmIiLCJ0eXAiOiJCZWFyZXIiLCJhenAiOiJJRFAiLCJzZXNzaW9uX3N0YXRlIjoiODJkYWM4MjMtMTViOS00MmVlLWE2YzEtZjg2ZDQ2MzY1ZjAzIiwiYWNyIjoiMSIsInNjb3BlIjoiIiwibmJmIjowLCJyb2xlIjoie1wiZ2VvLmlkcC5ub3RpZnlcIjpbXCJ1c2VyXCJdLFwiZ2VvLmFwcC5vYWRcIjpbXCJ1c2VyXCJdLFwiZ2VvLmlkcC5kYXRhc3RvcmVcIjpbXCJ1c2VyXCJdLFwiZ2VvLmFwcC53b3JrYmVuY2hcIjpbXCJ1c2VyXCJdfSIsInJvbGVzIjp7Imdlby5pZHAubm90aWZ5IjpbInVzZXIiXSwiZ2VvLmFwcC5vYWQiOlsidXNlciJdLCJnZW8uaWRwLmRhdGFzdG9yZSI6WyJ1c2VyIl0sImdlby5hcHAud29ya2JlbmNoIjpbInVzZXIiXX0sInN1aWQiOiIxMTg2NTEzNTQ2IiwidXVpZCI6Ijc5Yjg0MTZlLTY0NDYtNGMwYy1hODg1LWY1MzE2ZGMzOWMyZSIsImxvYSI6MTAwfQ.5W_E4fkhirbJZNAJ_TwMbLhcKdmnHBXOjvLUr4vW-DBRvSFfQrpdlDHLMIVI4B7bZ-OU_FVnH__i_diKYJFRH4l3Zqy8maa1pyj_WhZJksqBB69ehv8xx_3qtuJCZ0z0hln0FzmyG_Ep_uaru3gK_h33SuFxjdKr4F5XocyrYpGE-ewm-mBLj4DOBnZSJ4HgV0BG02LJIPIU8BybTmvgV-4mW3LXOVKDUJMmP4TF_ZEUzNz4a1vhoW4VIOvaNkk_8v8m_R4zjNOGmd_4jWEywORBZ1ofqvn72usY7TWEVpGBxR-rKYgzWXrdeBE4_l61MT420rBID9dbI2zRgEyVIQ","expires_in":14400,"refresh_expires_in":0,"token_type":"bearer","not-before-policy":0,"session_state":"82dac823-15b9-42ee-a6c1-f86d46365f03","scope":""}

Notice that the length of the content is 1500. That is also what the content-length: 1500 header says. When I do the same test using Python, I get the same result: a 1500 characters result.

But when I do the same test using Oracle UTL_HTTP, the result is only 1453 characters. Here is debugging from my PL/SQL code:

% resp.status_code=200
% resp.reason_phrase=OK
% resp.http_version=HTTP/1.0
% resp.get_headers
% .. Server: IIS
% .. Date: Thu, 25 Feb 2021 17:50:51 GMT
% .. Content-Type: application/json
% .. Content-Length: 1453
% .. X-Content-Type-Options: nosniff
% .. X-XSS-Protection: 1; mode=block
% .. Strict-Transport-Security: max-age=31536000; includeSubdomains;
% .. Cache-Control: no-store
% .. Set-Cookie: KC_RESTART=; Version=1; Expires=Thu, 01-Jan-1970 00:00:10 GMT;
Max-Age=0; Path=/auth/realms/IDP/; HttpOnly
% .. Pragma: no-cache
% .. X-Frame-Options: SAMEORIGIN
% .. Referrer-Policy: no-referrer
% .. Vary: Origin
% .. Via: 1.1 google
% .. Alt-Svc: clear
% Response:
{"access_token":"eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICIxMThSRENzZT
lqUWR4UVRnSkt2ZXlvSHBaaWE4R0pIVEU5RjJPSmE1M3N3In0.eyJleHAiOjE2MTQyODk4NTEsImlhd
CI6MTYxNDI3NTQ1MSwianRpIjoiMDQ4ZjhlMjctZTgwZi00MjIyLWFmNDAtMjZlNDdmYTFhMDg0Iiwi
aXNzIjoiaHR0cHM6Ly8zNS4xOTAuNTkuNzkvYXV0aC9yZWFsbXMvSURQIiwiYXVkIjoiSURQIiwic3V
iIjoiZjZiNGY1ZDQtMzM2Yi00ZTE3LTg3NzYtYzYwNWU3MzU0Y2JiIiwidHlwIjoiQmVhcmVyIiwiYX
pwIjoiSURQIiwic2Vzc2lvbl9zdGF0ZSI6ImYzYmZlOGJiLTFiZGYtNDQ5OS04NDQwLWIxODk5OGYwY
jg5NiIsImFjciI6IjEiLCJzY29wZSI6IiIsIm5iZiI6MCwicm9sZSI6IntcImdlby5pZHAubm90aWZ5
XCI6W1widXNlclwiXSxcImdlby5hcHAub2FkXCI6W1widXNlclwiXSxcImdlby5pZHAuZGF0YXN0b3J
lXCI6W1widXNlclwiXSxcImdlby5hcHAud29ya2JlbmNoXCI6W1widXNlclwiXX0iLCJyb2xlcyI6ey
JnZW8uaWRwLm5vdGlmeSI6WyJ1c2VyIl0sImdlby5hcHAub2FkIjpbInVzZXIiXSwiZ2VvLmlkcC5kY
XRhc3RvcmUiOlsidXNlciJdLCJnZW8uYXBwLndvcmtiZW5jaCI6WyJ1c2VyIl19LCJzdWlkIjoiMTE4
NjUxMzU0NiIsInV1aWQiOiI3OWI4NDE2ZS02NDQ2LTRjMGMtYTg4NS1mNTMxNmRjMzljMmUiLCJsb2E
iOjEwMH0.XHwxx3TzNNwgzVMv18Jav4bqXW9Q4n2bP_HV1iy0K4VPH-w84tXsHjXfH_f05Ynn2CXqv-
rdHds6KtuZaI1aypNnIvNvmbUiNHd6M1geLY4w8Yy9rg9-WFjYiFXbLTP7vvUAMSHueJmeT6WvzAsUT
Z7IQdp0w5aLDQ6ElV8pX1khBMCC7uXedRRDK-UC1MlJBrWtbhIMu5MaqpdpPeBcBMCvmqUBFTFfW6dQ
Ko01jeDjxePz_gZ2wdyU8fkV8UNTzkS3i6PYUkcxi3pmEC5r93JSNGVRUsZ53y5IjcaJK4aRXvvZQzV
iOitsbu8Pfciii2E_NDlk3qYgSqlxVrmzNA","expires_in":14400,"refresh_expires_in":0,
"token_type":"bearer","not-before-policy":0,"session_state":"f3bfe8bb-1bdf-4499
-8440-b18998f0b896","scope":""}

Notice the content length is now 1453 characters. The difference is with the token information inside the JSON response. It should be 1330 characters, but is only 1283 characters. The rest of the JSON document is the same. And the returned token is not valid for further use.

I can't find any explanation as to why the response is shorter when requested from UTL_HTTP. I first thought it had something to do with character set encoding. I have everything set to UTF-8. The token is returned in a base64 encoding.

Here is the code I use (I did not include the debugging code):

-- Setup the http request type and add the content
http_req := utl_http.begin_request(url, 'POST', 'HTTP/1.0');
utl_http.set_header(http_req, 'Content-Type', content_type);
utl_http.set_body_charset(http_req,'UTF-8');
utl_http.set_header(http_req, 'Content-Length', length(post_content));
utl_http.write_text(http_req, post_content);

-- Call the REST endpoint and fetch the http response
http_resp := utl_http.get_response(http_req);
utl_http.set_body_charset(http_resp,'UTF-8');

-- Read the response content
begin
  json_response := '';
  i := 0;
  loop
    utl_http.read_line(http_resp, response_line, false);
    json_response := json_response || response_line;
    i := i + 1;
  end loop;
exception
  when utl_http.end_of_body then
    utl_http.end_response(http_resp);
end;

I have been staring at this problem for hours. I tried various things, like setting or not setting explicit character set encoding, all to not effect. I can't see what I'm doing wrong and why Oracle would do anything with the response. I could imagine it truncating it for some reason - but why would characters be removed from inside the response ?

Oracle UTL_HTTP request returns a shorter response than when the same request is done in Python or curl

Answers (1)

Related Questions