CODEWITHSUNDEEP
pythonpython-requests
e.iluf
e.iluf

Reputation: 1659

How to prevent Python request from aborting after running

I'm running a python request to

url = 'https://www.walmart.com/store/1003-York-pa/search?query=ice%20cream'
api_url = 'https://www.walmart.com/store/electrode/api/search'

    params = {
        'query': word,
        'cat_id': 0,
        'ps': 24,
        'offset': 0,
        'prg': 'desktop',
        'stores': re.search(r'store/(\d+)', url).group(1)
    }

    data = requests.get(api_url, params=params).json()

I am trying to loop through thousands of query word. After running through couple of thousand of query request using the code the connection aborts and display this error message

data = requests.get(api_url, params=params).json() File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests/api.py", line 72, in get return request('get', url, params=params, **kwargs) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests/api.py", line 58, in request return session.request(method=method, url=url, **kwargs) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests/sessions.py", line 512, in request resp = self.send(prep, **send_kwargs) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests/sessions.py", line 622, in send r = adapter.send(request, **kwargs) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests/adapters.py", line 495, in send raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(54, 'Connection reset by peer'))

what is causing it to abort and how can I fix it?

Upvotes: 0

Views: 336

Answers (1)

Fabiano Teichmann
Fabiano Teichmann

Reputation: 1

It would appear that ECONNRESET means that the other side has closed the connection without reading.My suggestion for you is catch payload sender when the error occurred. And try again send payload. If you succeed in that case it may be instability in your network in prod.

Upvotes: 0

Related Questions