CODEWITHSUNDEEP
pythonethereumweb3pygo-ethereum
Initial Commit
Initial Commit

Reputation: 517

Intermittent "Read time out" errors when using web3.py to query remote ethereum node

I am trying to run some web3.py function calls to retrieve data from a remote Ethereum geth node running Rinkeby testnet hosted on an AWS EC2 Linux instance.

I set up my provider like this from local Python3 interpreter and it connects successfully and it works intermittently for certain basic function calls:

import web3, json, requests
from web3 import Web3, HTTPProvider
provider = HTTPProvider( 'http://remote-node-ip-address:8545' )
w3 = Web3(provider)

However, when I run certain function calls (like w3.eth.accounts from the Python3 interpreter), the remote server seems to slow down significantly (hang) and basically times out very often with this error:

requests.exceptions.ReadTimeout: HTTPConnectionPool(host='remote-node-ip', port=8545): Read timed out.

(read timeout=10)

But sometimes it works just fine, so the overall network connectivity is in place. When I SSH onto the remote AWS server which is actually a Docker container, and it does seem to be laggy and slow. The only thing I notice from the TOP output below is that the %CPU for WA is very high at 99.5%:

> top - 23:44:51 up  6:42,  0 users,  load average: 1.76, 1.73, 1.75
> Tasks:   4 total,   1 running,   3 sleeping,   0 stopped,   0 zombie
> %Cpu(s):  0.3 us,  0.3 sy,  0.0 ni,  0.0 id, **99.5 wa**,  0.0 hi,  0.0
> si,  0.0 st KiB Mem :  2049248 total,  1102520 free,   596396 used,  
> 350332 buff/cache KiB Swap:        0 total,        0 free,        0
> used.  1289532 avail Mem 
> 
>   PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+
> COMMAND         406 root      20   0 1526260 491008    424 S  0.5 24.0
> 0:05.30 geth         
>     1 root      20   0   56416  11620      0 S  0.3  0.6   1:18.18 supervisord     422 root      20   0   36636   1116    684 R  0.3  0.1
> 0:00.01 top             412 root      20   0   18232    460      8 S 
> 0.0  0.0   0:00.02 bash

I tried scaling up my AWS instance to a 4-vCPU, cpu-optimized c5.xlarge instance, but I had the same issue. I also tested the same commands against a local geth node running Rinkeby on my localhost, and there are no issues.

Does any one have any input on the best way to troubleshoot these issues with my remote geth node?

Upvotes: 1

Views: 753

Answers (1)

carver
carver

Reputation: 2349

Looks like you're maxed out on I/O. You could try to confirm with iotop.

Ethereum nodes run best on a local SSD. If you're trying to use EBS or even a local spinning disk, your node may grind to a halt regularly.

Check out the instances that support a local SSD:

The following instances support instance store volumes that use solid state drives (SSD) to deliver high random I/O performance: C3, F1, G2, I2, I3, M3, R3, and X1.

Upvotes: 1

Related Questions