CODEWITHSUNDEEP
hadoopblockchain
Atul Agrawal
Atul Agrawal

Reputation: 1520

How Blockchain is different from HDFS and how bitcoin mining is different from Map reduce or spark?

i am going through with blockchain technology and what i found is that it saves files on multiple different server.Now as per Hadoop documentation it is also stores files on multiple servers and keep 1 master copy on 1 master node. It also has zoo keeper which maintains the data between master node and all the secondry nodes.

Now my question is there any zookeeper for blockchain as well which is maintaing master copy or i am on the wrong way of understanding?

Upvotes: 6

Views: 5647

Answers (2)

Srini Sydney
Srini Sydney

Reputation: 580

blockchain is based on IPFS which is different from HDFS. IPFS is decentralized file system whereas HDFS is a distributed file system. In HDFS replication factor is constant but in IPFS you a peer of interest can replicate data.

Interesting paper where-in its described how the performance of hadoop jobs have increased using IPFS on top of HDFS Link - https://www.cse.unsw.edu.au/~hpaik/thesis/showcases/16s2/scott_brisbane.pdf

Upvotes: -2

Jim O'Donnell
Jim O'Donnell

Reputation: 66

These are fairly different animals. Most blockchain full nodes operate completely independently and spend their time validating transactions. The fact is that every full node in Bitcoin protocol or Ethereum has a full copy of the the entire chain--that is, every transaction that has ever occurred since block zero. They operate and validate on a consensus basis, so it is truly headless, not a distributed cluster.

HDFS is organized on a node (usually) as "just a bunch of disks" (JBOD), meaning they don't even usually bother with RAID. Replication is accomplished across nodes, but it's not full replication, meaning that there may be 5 copies of a file in a medium or large cluster and there is potential (which increases with the size of the cluster) for disk failures to cause actual data loss.

HDFS scales massively, but is inherently risky and requires a backup strategy. With blockchain, on the other hand, is much more compact, but since there are thousands or millions of complete copies of the ledger, each of which is improbably difficult to modify after the fact, it is virtually impossible to lose data.

Upvotes: 5

Related Questions