CODEWITHSUNDEEP
javalinuxhadoophdfs
Firechaser
Firechaser

Reputation: 25

Set space quota of HDFS in java

I am tryging to set space quota for HDFS using Hadoop Java API, however, I can only find getSpaceQuota method in ContentSummary class. Basically I want the code to achieve the same function as sudo -u hdfs hdfs dfsadmin -setSpaceQuota 1k /quotasdir command. Is there any API having set space quota method? Or any better idea? Thanks in advance.

Upvotes: 2

Views: 1296

Answers (1)

Chris Nauroth
Chris Nauroth

Reputation: 9854

In the Apache Hadoop codebase, the code for the hdfs dfsadmin -setSpaceQuota command is in the DFSAdmin class. If you read through that code, you'll see that it ultimately delegates to the DistributedFileSystem#setQuota method. That method implements an RPC to the NameNode to modify the quota.

If you want to reimplement this functionality in your own Java program, then you'll likely need to get an instance of FileSystem, downcast it to DistributedFileSystem, and then call DistributedFileSystem#setQuota.

Please be aware that Apache Hadoop does not provide a strong backward compatibility guarantee for the DistributedFileSystem class. That means that it would be possible for your code to break after an upgrade to a new Hadoop version. The class is annotated as LimitedPrivate and Unstable. The Apache Hadoop Compatibility documentation describes the meaning of these annotations in detail. There is currently no guaranteed public, stable API for changing quota from a custom program.

Upvotes: 2

Related Questions