Hadoop table source details

Question

I have a hadoop table and I need to find the source type(relational/flatfile/mainframe) of the table. I have access to hue, logs of table. May I know how can I extract those details?

Answer 1

You cannot find the source of the data from a hive table. Hive/Impala tables only holds the information and metadata about the data. It doesn't holds where it came from. For e.g. if you are loading the data from a mainframe system using Sqoop, then you can load the data into a hadoop system in avro, parquet, compressed, delimited, etc. or into a hive table.

If you really want to understand the lineage of your data, you need to look for enterprise solutions like Cloudera Navigator. The discussion of this topic is too broad for this forum.

Cloudera Navigator Lineage URL : https://docs.cloudera.com/documentation/enterprise/5-3-x/topics/cn_iu_lineage.html