CODEWITHSUNDEEP
pythonsshgeventssh-tunnelgrequests
David
David

Reputation: 55

Conflict between grequests / gevent and sshtunnel

I'm working on a web scraper that makes use of grequests, and then sshtunnel to store the data in a database.

However importing grequests and sshtunnel causes Error reading SSH protocol banner cannot switch to a different thread

If grequests is imported nothing will work, without it being imported the sshtunnel works perfectly. It's not possible to remove grequests. I've tried with and without monkeypatch, the result is the same.

from gevent import monkey
monkey.patch_all(thread=False, select=False)
import grequests
import mysql.connector
import sshtunnel

I've found a small number of mentions online about conflicts between gevent and paramiko (sshtunnel) but no solutions.

Paramiko does not work inside Django when grequests is loaded

pysftp, paramiko, grequests: Error reading SSH protocol banner

However none of these are exactly the same error, and none of them solve the problem.

Has anyone encountered this and/or found a workaround?

I don't need any parallel ssh connections in the code, just one connection at the start to load current data, and one at the end to dump the new data. This problem happens as soon as grequests is imported, it doesn't have to be used anywhere.

Upvotes: 1

Views: 75

Answers (0)

Related Questions