CODEWITHSUNDEEP
azurehadoophiveazure-hdinsight
Srinivas
Srinivas

Reputation: 2539

HDInsight Azure Blob Storage Change

On HDInsight cluster, a Hive table is created using CREATE EXTERNAL statement:

CREATE EXTERNAL TABLE HTable(t1 string, t2 string, t3 string) ROW FORMAT DELIMITED FIELDS TERMINATED BY ' ' STORED AS TEXTFILE LOCATION 'wasb://$containerName@$storageAccountName.blob.core.windows.net/HTable/data/';

Then some existing files changed, some files are added to Azure Blob Container mentioned in the CREATE statement.

Does a new hive query consider changes made to Blob Container with out again loading data to hive table?

Upvotes: 0

Views: 601

Answers (1)

Remus Rusanu
Remus Rusanu

Reputation: 294277

Yes, your table definition is saved in the Hive metastore. You can subsequently simply query HTable and data will be there. Normally Hive on HDInsight follows the same rules that applies to Hive and HDFS.

For a more advanced discussion you can play some tricks, but you need to know what you're doing. Because HDInsight storage can survive a cluster lifetime, with HDInsight is feasible to tear down the cluster and redeploy a new HDInsight cluster and still have the Hive data. You can even keep the Hive metastore, as is a separate database (an SQL Azure DB). With an HDFS based cluster a recycle of the cluster leads to loss of all HDFS data.

Upvotes: 1

Related Questions