CODEWITHSUNDEEP
hadoopapache-pighadoop2
Hadoop_Beginner
Hadoop_Beginner

Reputation: 31

Apache PIG - How to change the standard output name "part-r-00000" of files?

I have a .pig script which creates a file with some calculated data. I want to have the output file named "result.txt" instead of the meaningless standard output name "part-r-00000".

The last entry in my .pig script is

store C into 'result' USING PigStorage();

Therefore "result" is the name of the folder in HDFS where the result.txt should be stored.

How can I do this?

Upvotes: 2

Views: 990

Answers (1)

mr2ert
mr2ert

Reputation: 5184

The part-r-XXXXX are not meaningless depending on how you plan to use them. If you need to load the results of this pig script into another pig script you can just do:

A = LOAD 'result' USING PigStorage() AS (...) ;

As LOAD can take a directory as input and will load each file in that directory.

If you need to handle the output (locally) as a single text file, then there is nothing builtin to PIG that will accomplish that. You will need to write a script to pull the results from hdfs and concatenate all of the part-r-XXXXX files together.

Upvotes: 1

Related Questions