CODEWITHSUNDEEP
hadoophivehiveql
Karnimrod
Karnimrod

Reputation: 425

What is the difference between -hivevar and -hiveconf?

From hive -h :

--hiveconf <property=value>   Use value for given property
--hivevar <key=value>         Variable subsitution to apply to hive
                                  commands. e.g. --hivevar A=B

Upvotes: 25

Views: 38530

Answers (6)

MonaPy
MonaPy

Reputation: 349

I found some differences between hiveconf and hivevar with hive/beeline circa 2023.

hive> set param1= 123; -- does not work
hive> set hiveconf:param1 = 123; -- does not work
hive> set hivevar:param1 = 123; -- works
hive> select ${hivevar:param1}; -- shows 123

In the first 2 cases, the error is:

Cannot modify parame1 at runtime. It is not in list of params that are allowed to be modified at runtime (state=42000,code=1)

The difference apparently come from the the system hive/beeline conf:

hive.security.authorization.sqlstd.confwhitelist  
OR
hive.security.authorization.sqlstd.confwhitelist.append

In addition, as mentioned by another post, hiveconf vars are meant to be used for system-wide hive config, while hivevar is used for user config. How to set variables in HIVE scripts

Upvotes: 1

charan tej
charan tej

Reputation: 1054

You can refer this for the difference

https://cwiki.apache.org/confluence/display/Hive/LanguageManual+VariableSubstitution

There are three namespaces for variables – hiveconf, system, and env. (Custom variables can also be created in a separate namespace with the define or hivevar option in Hive 0.8.0 and later releases.)

Upvotes: 1

RPaul
RPaul

Reputation: 179

We can also use them at the beginning of the script as:

hiveconf:

SET this_dt = CURRENT_DATE;
select ${hiveconf:this_dt};

hivevar:

set hivevar:cur_dt=current_date;
select ${hivevar:cur_dt};

Upvotes: 5

Rahul Sharma
Rahul Sharma

Reputation: 5834

@Llama has explained it in detailed, along with that both type of variables are accessed differently.

The --hivevar variables are accessed using ${var-name}, while the --hiveconf are accessed ${hiveconf:var-name} inside hive.

e.g. Below examples access variable and print it's value in hive.

hivevar:

hive --hivevar a='this is a' -e '!echo ${a};'

output:this is a

hiveconf:

hive --hiveconf a='this is a' -e '!echo ${hiveconf:a};'

output:this is a

Upvotes: 15

Mr. Llama
Mr. Llama

Reputation: 20909

I didn't quite feel like the examples from the documentation were adequate, so here's my attempt at an answer.

In the beginning there was only --hiveconf and variable substitution didn't exist.

The --hiveconf option allowed users to set Hive configuration values from the command line and that was it. All Hive configuration values are stored under the hiveconf namespace, i.e. hiveconf:mapred.reduce.tasks. These values allowed you to control things like the number of mappers and reducers, if status messages should be displayed, and if the script should continue on errors.

Later, variable substitution was added. This meant you could now use variables in queries with the ${...} syntax. However, the only variables you could set from the command line were under the hiveconf namespace using --hiveconf, so that's where users put their variables.

Putting your personal variables under the Hive configuration namespace probably won't break anything, but it's also not good form. Later, it was suggested that a hivevar namespace be added specifically for user variables which could also be defined at the command line using --hivevar. This meant a cleaner separation between Hive configuration values and user defined variables.

In summary:
The hiveconf namespace and --hiveconf should be used to set Hive configuration values.
The hivevar namespace and --hivevar should be used to define user variables.
Setting user variables under the hiveconf namespace probably won't break anything, but isn't recommended.

Upvotes: 53

leftjoin
leftjoin

Reputation: 38335

No difference except the namespace. hiveconf and hivevar are different namespaces. hivevar namespace was added just to separate configuration properties namespace and Hive variables namespace. See https://issues.apache.org/jira/browse/HIVE-2020 for more information.

Upvotes: 0

Related Questions