CODEWITHSUNDEEP
hadoophivehadoop-streaming
Mark Vickery
Mark Vickery

Reputation: 1955

Pass environment variables to Hive Transform or MapReduce

I am trying to pass a custom environment variable to an executable (my-mapper.script in the example below) used in a Hive Transform eg:

SELECT
   TRANSFORM(x, y, z)
   USING 'my-mapper.script'
FROM
(
   SELECT
      x, y, z
   FROM
      table
)

I know in Hadoop streaming this can be achieved using 

-cmdenv EXAMPLE_DIR=/home/example/dictionaries/

But I do not know how to do this in a Hive Transform/MapReduce.

Any ideas?

Upvotes: 2

Views: 2065

Answers (2)

libjack
libjack

Reputation: 6443

You can wrap your script with a simple 2 line bash script to setup the environment. e.g

#!/bin/sh
export FOO=boo
my-mapper.script

And then use this script in the query

USING 'wrapper.sh'

my-mapper.script will see FOO (with value "boo") in the environment.

Upvotes: 2

Dan Ciborowski - MSFT
Dan Ciborowski - MSFT

Reputation: 7207

Are you looking for something like this?

% hive -hiveconf CURRENT_DATE='2012-09-16' -f test.hql

Upvotes: 0

Related Questions