CODEWITHSUNDEEP
hadoophivehdfs
user1651157
user1651157

Reputation:

How to view the hadoop data directory structure?

I have partitioned table in hive. So I wanna see the directory structure in hadoop hdfs?

From documentation, I have found the following command

hadoop fs -ls /app/hadoop/tmp/dfs/data/

and /app/hadoop/tmp/dfs/data/ is my data path. But this command return

ls: Cannot access /app/hadoop/tmp/dfs/data/: No such file or directory.

Am I missing something there?

Upvotes: 0

Views: 15974

Answers (2)

Charles Menguy
Charles Menguy

Reputation: 41458

Unless I'm mistaken, it seems you are looking for a temporary directory that you probably defined in the property hadoop.tmp.dir. This is a local directory, but when you do hadoop fs -ls you are looking at what files are available in HDFS, so you won't see anything.

Since you're looking or the Hive directories, you are looking for the following property in your hive-site.xml: 

hive.metastore.warehouse.dir

The default is /user/hive/warehouse, so if you haven't changed this property you should be able to do:

hadoop fs -ls /user/hive/warehouse

And this should show you your table directories.

Upvotes: 1

Sasi Kumar M
Sasi Kumar M

Reputation: 2630

check whether tmp directory is correctly set in your core-site.xml file and hdfs-site.xml. if not set, then the temporary directory of operating system(tmp in ubuntu and %temp% in windows) will be set to hadoop tmp folder, due to which you may lose your data after restarting your computer. Set this dfs.tmp.dir in both the xml and restart your cluster. It will work fine then. even after this if it is not resolved, please give more details about partitioning table code and the table data too.

Upvotes: 0

Related Questions