How to get around python requests SSL and proxy error?

Question

When sending a request with authentication, I get a requests.exceptions.SSLError error which you can See below.

proxies = { 'https' : "http://user:pass@ip:port/" } 

url = "https://httpbin.org/ip"

numberResponse = requests.get(url,proxies=proxies).text

print(numberResponse)

The requests.exceptions.SSLError

Traceback (most recent call last):
  File "C:\Users\K_Yuk\AppData\Local\Programs\Python\Python38\lib\site-packages\urllib3\connectionpool.py", line 696, in urlopen       
    self._prepare_proxy(conn)
  File "C:\Users\K_Yuk\AppData\Local\Programs\Python\Python38\lib\site-packages\urllib3\connectionpool.py", line 964, in _prepare_proxy
    conn.connect()
  File "C:\Users\K_Yuk\AppData\Local\Programs\Python\Python38\lib\site-packages\urllib3\connection.py", line 359, in connect
    conn = self._connect_tls_proxy(hostname, conn)
  File "C:\Users\K_Yuk\AppData\Local\Programs\Python\Python38\lib\site-packages\urllib3\connection.py", line 496, in _connect_tls_proxy
    return ssl_wrap_socket(
  File "C:\Users\K_Yuk\AppData\Local\Programs\Python\Python38\lib\site-packages\urllib3\util\ssl_.py", line 428, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(
  File "C:\Users\K_Yuk\AppData\Local\Programs\Python\Python38\lib\site-packages\urllib3\util\ssl_.py", line 472, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
  File "C:\Users\K_Yuk\AppData\Local\Programs\Python\Python38\lib\ssl.py", line 500, in wrap_socket
    return self.sslsocket_class._create(
  File "C:\Users\K_Yuk\AppData\Local\Programs\Python\Python38\lib\ssl.py", line 1040, in _create
    self.do_handshake()
  File "C:\Users\K_Yuk\AppData\Local\Programs\Python\Python38\lib\ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:1125)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\K_Yuk\AppData\Local\Programs\Python\Python38\lib\site-packages\requests\adapters.py", line 439, in send
    resp = conn.urlopen(
  File "C:\Users\K_Yuk\AppData\Local\Programs\Python\Python38\lib\site-packages\urllib3\connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "C:\Users\K_Yuk\AppData\Local\Programs\Python\Python38\lib\site-packages\urllib3\util\retry.py", line 573, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='httpbin.org', port=443): Max retries exceeded with url: /ip (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1125)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:/Users/K_Yuk/OneDrive/Desktop/Gmail generator/test.py", line 15, in <module>
    numberResponse = requests.get(url,proxies=proxies).text
  File "C:\Users\K_Yuk\AppData\Local\Programs\Python\Python38\lib\site-packages\requests\api.py", line 76, in get
    return request('get', url, params=params, **kwargs)
  File "C:\Users\K_Yuk\AppData\Local\Programs\Python\Python38\lib\site-packages\requests\api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Users\K_Yuk\AppData\Local\Programs\Python\Python38\lib\site-packages\requests\sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Users\K_Yuk\AppData\Local\Programs\Python\Python38\lib\site-packages\requests\sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "C:\Users\K_Yuk\AppData\Local\Programs\Python\Python38\lib\site-packages\requests\adapters.py", line 514, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='httpbin.org', port=443): Max retries exceeded with url: /ip (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1125)')))

So then I tired verify=False as one of the requests.get() parameters but then get a requests.exceptions.ProxyError error which you can see below :

proxies = { 'https' : "http://user:pass@10.10.1.10:3128/"} 

url = "https://httpbin.org/ip"

numberResponse = requests.get(url,proxies=proxies,verify=False).text

print(numberResponse)

The requests.exceptions.ProxyError

Traceback (most recent call last):
  File "C:\Users\K_Yuk\AppData\Local\Programs\Python\Python38\lib\site-packages\urllib3\connectionpool.py", line 696, in urlopen       
    self._prepare_proxy(conn)
  File "C:\Users\K_Yuk\AppData\Local\Programs\Python\Python38\lib\site-packages\urllib3\connectionpool.py", line 964, in _prepare_proxy
    conn.connect()
  File "C:\Users\K_Yuk\AppData\Local\Programs\Python\Python38\lib\site-packages\urllib3\connection.py", line 359, in connect
    conn = self._connect_tls_proxy(hostname, conn)
  File "C:\Users\K_Yuk\AppData\Local\Programs\Python\Python38\lib\site-packages\urllib3\connection.py", line 496, in _connect_tls_proxy
    return ssl_wrap_socket(
  File "C:\Users\K_Yuk\AppData\Local\Programs\Python\Python38\lib\site-packages\urllib3\util\ssl_.py", line 428, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(
  File "C:\Users\K_Yuk\AppData\Local\Programs\Python\Python38\lib\site-packages\urllib3\util\ssl_.py", line 472, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
  File "C:\Users\K_Yuk\AppData\Local\Programs\Python\Python38\lib\ssl.py", line 500, in wrap_socket
    return self.sslsocket_class._create(
  File "C:\Users\K_Yuk\AppData\Local\Programs\Python\Python38\lib\ssl.py", line 1040, in _create
    self.do_handshake()
  File "C:\Users\K_Yuk\AppData\Local\Programs\Python\Python38\lib\ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
FileNotFoundError: [Errno 2] No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\K_Yuk\AppData\Local\Programs\Python\Python38\lib\site-packages\requests\adapters.py", line 439, in send
    resp = conn.urlopen(
  File "C:\Users\K_Yuk\AppData\Local\Programs\Python\Python38\lib\site-packages\urllib3\connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "C:\Users\K_Yuk\AppData\Local\Programs\Python\Python38\lib\site-packages\urllib3\util\retry.py", line 573, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='httpbin.org', port=443): Max retries exceeded with url: /ip (Caused by ProxyError('Cannot connect to proxy.', FileNotFoundError(2, 'No such file or directory')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:/Users/K_Yuk/OneDrive/Desktop/Gmail generator/test.py", line 15, in <module>
    numberResponse = requests.get(url,proxies=proxies,verify=False).text
  File "C:\Users\K_Yuk\AppData\Local\Programs\Python\Python38\lib\site-packages\requests\api.py", line 76, in get
    return request('get', url, params=params, **kwargs)
  File "C:\Users\K_Yuk\AppData\Local\Programs\Python\Python38\lib\site-packages\requests\api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Users\K_Yuk\AppData\Local\Programs\Python\Python38\lib\site-packages\requests\sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Users\K_Yuk\AppData\Local\Programs\Python\Python38\lib\site-packages\requests\sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "C:\Users\K_Yuk\AppData\Local\Programs\Python\Python38\lib\site-packages\requests\adapters.py", line 510, in send
    raise ProxyError(e, request=request)
requests.exceptions.ProxyError: HTTPSConnectionPool(host='httpbin.org', port=443): Max retries exceeded with url: /ip (Caused by ProxyError('Cannot connect to proxy.', FileNotFoundError(2, 'No such file or directory')))

I tired to look every for the answer but nothing seems to work. I can't send a request with a proxy with authentication. Any ideas?

Answer 1

I had a similar problem with my script in python 3.9 version. When I try to get some data from web server, I had exception like Your:

urllib3.exceptions.MaxRetryError

Simple add this line to configure Your requests

import requests
import urllib3
requests.packages.urllib3.util.ssl_.DEFAULT_CIPHERS = 'ALL:@SECLEVEL=1'

I hope, this help You to solve Your issue

Answer 2

The problem is very likely not the authentication. Unfortunately, you don't provide details of the proxy configuration and the URL you use for the proxy. The only thing you provide is:

proxies = { 'https' : eampleIpWithAuth } 

Based on the reference to _connect_tls_proxy in the stacktrace the eampleIpWithAuth is very likely something like https://..., i.e. you try to access the proxy itself over HTTPS. Note that accessing a proxy over HTTPS is different from using a HTTP proxy for HTTPS. When accessing a HTTPS URL over a HTTPS proxy one essentially does double encryption to the proxy:

client --- [HTTPS wrapped inside HTTPS] --- proxy --- [HTTPS] --- server

Whereas with a HTTPS URL over a "normal" HTTP proxy there is only single encryption, i.e. it looks (simplified) like this:

client --- [HTTPS wrapped inside HTTP]  --- proxy --- [HTTPS] --- server

Very likely the proxy you want to use is a plain HTTP proxy, and not a HTTPS proxy. This is actually the most common case.

The error happens since the proxy is not able to speak TLS but gets accessed by TLS. The fix is to use http://proxy and not https://proxy as the proxy address. Note that the latter worked in older versions of Python since proxy over HTTPS was not supported and a value of https:// for the protocol was treated the same as http://.