CODEWITHSUNDEEP
python-3.xpython-requests
briba
briba

Reputation: 2987

Python requests too slow compared to Postman or cURL

I am trying to make a single API call using different approaches in Python3 but they are all incredibly slow if I compare with the same request in Postman and/or cURL.

This is what I am trying:

headers = {
   "Authorization": "Bearer " + self.access_token,
   "Content-Type": "application/json",
   "Accept-Encoding": "gzip, deflate, br",
   "Connection": "keep-alive",
   "Accept": "*/*",
   "User-Agent": "PostmanRuntime/7.28.2",
   "Cache-Control": "no-cache"
}

session = requests.Session()
api_res = session.get(self.api_endpoint, headers=headers, timeout=self.req_timeout,)

When running this call, it gets stuck for a few minutes until I receive a response. If I use Postman for example, I get the result in 1 second or less.

I also tried using http.client, urllib3 but I still see a huge delay in the call.

I also tried debugging the call:

import http.client

http.client.HTTPConnection.debuglevel = 1
logging.basicConfig()
logging.getLogger().setLevel(logging.DEBUG)
requests_log = logging.getLogger("requests.packages.urllib3")
requests_log.setLevel(logging.DEBUG)
requests_log.propagate = True

Debug response:

[DEBUG] Starting new HTTPS connection (1): ...:443

header: Content-Type: application/json; charset=utf-8
header: Content-Length: 61
header: Connection: keep-alive
header: Date: ...
header: Cache-Control: no-cache
header: Pragma: no-cache
header: Expires: -1
header: WWW-Authenticate: Bearer
header: Access-Control-Allow-Origin: *
header: P3P: CP="IDC DSP COR ADM DEVi TAIi PSA PSD IVAi IVDi CONi HIS OUR IND CNT"
header: Access-Control-Allow-Methods: GET, POST, OPTIONS, HEAD
header: X-XSS-Protection: 1; mode=block
header: Access-Control-Allow-Headers: accept, authorization, content-type, Cache-Control, P3P, GE-IGNORE-CACHE, Signature, fromMobile, ssoToken, fromAdmin, fromGameapp, fromGtv, fromGameapp, ManagerOrgUnitUserName, ManagerOrgUnitId, g-s-x, g-s-x-t, g-s-i-i, User-Agent, Referer, Origin, Access-Control-Allow-Headers
header: X-Content-Type-Options: nosniff
header: X-XSS-Protection: 1; mode=block
header: Referrer-Policy: same-origin
header: Strict-Transport-Security: max-age=31536000; includeSubDomains
header: X-Cache: Error from cloudfront
header: Via: 1.1 ....cloudfront.net (CloudFront)
header: X-Amz-Cf-Pop: ORD52-C1
header: X-Amz-Cf-Id: ...

Any ideas about why is it so slow? What it doesn't happen when I replicate to Postman for example?

Even from Google is taking a lot of time:

requests.get("https://www.google.com")

I also realized that it's working with ipv4 and NOT with ipv6.

Thanks!

Upvotes: 2

Views: 620

Answers (1)

briba
briba

Reputation: 2987

I found out that IPv6 was not working but IPv4 was. I had to force calls to IPv4 like this:

import socket
import requests.packages.urllib3.util.connection as urllib3_cn

urllib3_cn.allowed_gai_family = lambda: socket.AF_INET

Upvotes: 1

Related Questions