CODEWITHSUNDEEP
ethereumparity-io
IntegerSynergy
IntegerSynergy

Reputation: 21

Setting up a high-capacity Ethereum client

I'm looking to set up a server (or collection of servers) that can act as an Ethereum node to which I can send a large number of requests, looking at up to 100 per second, to get data from the blockchain such as account balances, transactions etc (Like Etherscan). Correct me if I'm wrong, but I don't think such a system could be possible with a regular parity or geth client running on a single server with the chain data on an SSD, so what I'm thinking of doing is the follwing:

  1. Configure a parity client with SSD on Server A, which will act as a regular node
  2. Have the chain data synchronized onto another SSD on Server B
  3. Set up parity client on Server B that is not connected to the Ethereum network, and uses the chain data copied from Server A. Does not process the transactions.
  4. Transactions, balances, etc. can be queried at Server B RPC socket.

The reason for this setup is so that Server B has little overhead as possible, that way it can handle many more queries.

I'm wondering if it makes sense to use this set up, or if it's even possible. A simpler solution would be best, but I can't find much info on how high capacity blockchain services run. I'd really appreciate any pointers.

Upvotes: 1

Views: 946

Answers (2)

q9f
q9f

Reputation: 11824

Correct me if I'm wrong, but I don't think such a system could be possible with a regular Parity or Geth client running on a single server with the chain data on an SSD.

I'm happy to correct you: Parity can easily handle 1_000's of RPC requests per second on consumer-grade hardware.

If you need 10_000's of requests per second, you are still able to achieve this with a single Parity instance on a high-end enterprise server with 128GB+ RAM and a Raid-0 redundant flash storage, make sure to configure a huge cache size with Parity:

parity --cache-size 65536

You could even go further in optimizations by putting the whole blockchain in memory by setting the --data-dir to a tmpfs.

If you still need more, I would run multiple setups like those described above and put a load balancer in front of the setup that directs the requests sorted by type to the available machines in the back.

Disclosure: I work for Parity. I think similar stats are true for Geth, however.

Upvotes: 2

mortimr
mortimr

Reputation: 111

I think that the best option would be block pre-processing. Run an instance of an Ethereum client, iterate on all blocks and pre-process them (extract all the informations you need and save it in a database). This will allow more flexibility on your requests.

Let’s say you want a list of transactions from a specific account. This can’t be done with current ETH RPC API (I mean not in an optimized way). The best thing would be to get all blocks pre-processed, extract all the transactions and arrange them in a database in order for you to be able to query for transactions coming from a specific address.

Running an API on top of the database you built would make your server way more performant.

Upvotes: 1

Related Questions