CODEWITHSUNDEEP
hadoopapache-pigcloudera
Hinchy
Hinchy

Reputation: 683

Apache Pig will not parse string to int/long

I'm new to pig and am trying to perform some basic analysis on a file containing events that look like the below:

1345477765  2012-08-20  08:49:24    servername  12.34.56.78 192.168.1.4 joebloggs   ManageSystem    Here's your message

I attempt to load the file as below:

logs = LOAD '/path/to/file' using PigStorage AS (loggedtime:long, serverdate:chararray, servertime:chararray, servername:chararray, externalip:chararray, internalip:chararray, username:chararray, systemtype:chararray,  message:chararray);

When I illustrate logs everything looks ok:

     Illustrate logs
    ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    | logs     | loggedtime:long      | serverdate:chararray    | servertime:chararray    | servername:chararray    | externalip:chararray        | internalip:chararray      | username:chararray    | systemtype:chararray            | message:chararray                                                                                                                         | 
    ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    |          | 1345477765 | 2012-08-20   | 08:49:24       | servername | 12.34.56.78 | 192.168.1.4 | joebloggs   | ManageSystem | Here's your message  | 
    ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Also, when a describe them everything is as I would expect:

logs: {loggedtime: long,serverdate: chararray,servertime: chararray,servername: chararray,externalip: chararray,internalip: chararray,username: chararray,systemtype: chararray,message: chararray}

However, when I dump logs, the loggedtime is not included.

dump logs;
(,2012-08-20,08:49:24,servername,12.34.56.78,192.168.1.4,joebloggs,ManageSystem,Here's your message)

Presumably as a result of this, my filter returns no events:

specificlog = FILTER logs BY loggedtime == 1345477765;

Hopefully I'm missing something easy here.

Upvotes: 1

Views: 1692

Answers (1)

Hinchy
Hinchy

Reputation: 683

I eventually figured this out myself. To parse to a long I had to put an "L" at the end of the number.

e.g. by changing my source data to the below I was able to get this working.

1345477765L  2012-08-20  08:49:24    servername  12.34.56.78 192.168.1.4 joebloggs   ManageSystem    Here's your message

Hopefully this will help someone with the same problem.

Upvotes: 2

Related Questions