CODEWITHSUNDEEP
pythonsslpython-requests
physicalattraction
physicalattraction

Reputation: 6858

Requests ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:777)

I am trying to get image data from a particular url (see code below). Their security is pretty outdated (see SSL report below), but I need to connect to it anyway. I am able to fetch the image using my browser.

This is what I try:

import requests
url = 'https://www.bestseller.com/webseller/psp.show_picture?picturesId=2367737&thumb=false'
requests.get(url)

The error I get is:

...
    File "/path/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 689, in do_handshake
self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:777)

I get the same error when I add the argument verify=False.

I have installed requests using pip install requests[security]. This is the relevant pip freeze output:

asn1crypto==0.24.0
certifi==2017.11.5
cryptography==2.1.4
ndg-httpsclient==0.4.3
pyasn1==0.4.2
pyOpenSSL==17.5.0
requests==2.8.1
urllib3==1.22

Other configuration prints:

>>> import ssl
>>> print(ssl.OPENSSL_VERSION)
OpenSSL 1.0.2n  7 Dec 2017
>>> from cryptography.hazmat.backends.openssl.backend import backend
>>> print(backend.openssl_version_text())
OpenSSL 1.1.0g  2 Nov 2017

Is there a way to disable setting up / verifying SSL, or if not, how do I find out which cipher I need to add and how do I do this?

When I try to write my own adapter:

from requests.adapters import HTTPAdapter
from requests.packages.urllib3.util.ssl_ import create_urllib3_context

# This is the 2.11 Requests cipher string, containing 3DES.
CIPHERS = (
    'ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+HIGH:'
    'DH+HIGH:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+HIGH:RSA+3DES:!aNULL:'
    '!eNULL:!MD5'
)

class DESAdapter(HTTPAdapter):
    def init_poolmanager(self, *args, **kwargs):
        context = create_urllib3_context(ciphers=CIPHERS)
        kwargs['ssl_context'] = context
        return super(DESAdapter, self).init_poolmanager(*args, **kwargs)

s = requests.Session()
s.mount(url, DESAdapter())

I get the following error:

AttributeError: 'super' object has no attribute 'init_poolmanager'

https://www.ssllabs.com/ssltest/analyze.html?d=www.bestseller.com

Upvotes: 3

Views: 6595

Answers (1)

physicalattraction
physicalattraction

Reputation: 6858

In the meantime, I have circumvented the issue using library pycurl.

import urllib.request
from io import BytesIO
from urllib.error import URLError

import pycurl
from django.core.files import File
from django.core.files.base import ContentFile


def _create_attachment_from_url(url: str):
    """
    Fetch the image from the given URL and save it as an attachment
    """

    try:
        file = _fetch_file_using_urllib(url)
    except URLError:
        file = _fetch_file_using_pycurl(url)

    return file


def _fetch_file_using_urllib(url: str) -> (File, str):
    response = urllib.request.urlretrieve(url)
    contents = open(response[0], 'rb')
    file = File(contents)
    return file


def _fetch_file_using_pycurl(url: str) -> (File, str):
    buffer = BytesIO()
    c = pycurl.Curl()
    try:
        c.setopt(c.URL, url)
        c.setopt(c.WRITEDATA, buffer)
        c.perform()
        contents = buffer.getvalue()
        file = ContentFile(contents)
    finally:
        c.close()
    return file

Upvotes: 0

Related Questions