CODEWITHSUNDEEP
hadoopapache-pig
Romain Jouin
Romain Jouin

Reputation: 4848

Hadoop Pig - how to set the directory for intermediate results

My Pig jobs are storing intermediate result on my hard drive, which is small. I would like Pig to store these intermediate results on /data. Which setting do I have to change?

Upvotes: 0

Views: 284

Answers (1)

Romain Jouin
Romain Jouin

Reputation: 4848

I had to add the following export in the pig conf file ( /pig.-0.14.0/conf/pig-env.sh): 

export PIG_OPTS="$PIG_OPTS -Dhadoop.tmp.dir=$path_to_big_hard_drive -Dpig.temp.dir=$path_to_big_hard_drive"

Upvotes: 1

Related Questions