CODEWITHSUNDEEP
hadoophdfs
Prashanth Chandra
Prashanth Chandra

Reputation: 2692

If we increase the size of files stored in HDFS without increasing the number of files, does the memory required by the NameNode increase or decrease?

I got the question (Q25) from https://www.tutorialspoint.com/hadoop/hadoop_mock_test.htm?min=65&max=96

Q 25 - If we increase the size of files stored in HDFS without increasing the number of files, then the memory required by namenode

A - Decreases

B - Increases

C - Remains unchanged

D - May or may not increase

In the answer key available here, it says that the memory required by the NameNode decreases but I fail to see how this is the case.

According to https://www.cloudera.com/documentation/enterprise/latest/topics/admin_nn_memory_config.html#concept_r14_qpc_r5
Each namespace object takes 150 bytes, and a namespace object is stored for every file and every block. E.g. file that is 3 blocks in size takes 4 namespace objects on the NameNode.

Thus, memory taken by namespace objects = (nFiles + nBlocks) * 150

If this is the case, shouldn't memory required by the NameNode increase? Since nFiles is constant while nBlocks increases.

Upvotes: 0

Views: 3191

Answers (1)

Brian Gallew
Brian Gallew

Reputation: 41

If we assume their answer is correct, then we must also assume that what we're really changing is the block size. If the block size increases, then fewer blockes are required to hold the same storage.

Upvotes: 1

Related Questions