CODEWITHSUNDEEP
hadoopapache-pighdfs
RKCY
RKCY

Reputation: 4135

how to load the data from local system to hdfs using PIG

I have a csv file sample.csv and located in \home\hadoop\Desktop\script\sample.csv . I tried to load in PIG using 

movies = load '/home/hadoop/Desktop/script/sample.csv' using PigStorage(',') as (id,name,year,rating,duration);

But this PIG statement is giving an error but while giving statement as dump movies;, it is throwing error and showing input and output is failed.

Please suggest me how to load the data using pig statement.

Upvotes: 2

Views: 10845

Answers (3)

swapnil kolapkar
swapnil kolapkar

Reputation: 25

open pig shell in local mode by pig -x local and if your file present at hdfs then you can use pig to open grant shell.

$pig -x local
grunt> movies = load '/home/hadoop/Desktop/script/sample.csv' using PigStorage(',') as (id:int,name:chararray,year:chararray,rating:chararray,duration:chararray);


grunt> dump movies;

Upvotes: 0

Rashid Ali
Rashid Ali

Reputation: 192

You can also use copyFromLocal command in grunt shell to move local file to hdfs.

Upvotes: 1

Surender Raja
Surender Raja

Reputation: 3599

If your input file is at local then you can enter into grunt shell by typing pig -x local

If you enter into grunt shell then you can type the below statement

 record = LOAD  '/home/hadoop/Desktop/script/sample.csv' using PigStorage(',') as (id:int,name:chararray,year:chararray,rating:chararray,duration:int); 


dump record;

If your input file is not at local then first you need to copy that file from local to HDFS using below command

hadoop dfs -put <path of file at local>  <path of hdfs dir>

Once your file is loaded into HDFS you can enter to map reduce mode by typing pig

again grunt shell will be opened. ia assuming that your HDFS location is something like below LOAD statement

record = LOAD  '/user/hadoop/inputfiles/sample.csv' using PigStorage(',') as (id:int,name:chararray,year:chararray,rating:chararray,duration:int); 


dump record;

Upvotes: 2

Related Questions