CODEWITHSUNDEEP
sqlstringdatehivehiveql
sai
sai

Reputation: 323

string to date - hive SQL

I am running queries in a hive environment.

I have a column which has a timestamp but is set up a string in the tables. I tried the following : all of them return Null 

SELECT 
,To_date(activitydate)

Cast:

,cast(activitydate as timestamp)

This is the how the data is set up in the table:

Appreciate any inputs on how I can convert this :

05/12/2017 00:00:00     



SELECT 
 cust_id
,to_date(activitydate)  activity_date
,type type_of_contact


FROM repl_task

WHERE to_date(activitydate) BETWEEN '2014-01-01' AND '2017-01-01' ;

I am running out of memory if I run this :

SELECT 
 cust_id
,activitydate
,SUBSTR(activitydate,4,2) AS DT
,SUBSTR(activitydate,0,2) AS MONTH 
,SUBSTR(activitydate,7,4) AS YEAR
,type 


FROM task 

WHERE activitydate >='01/01/2016 00:00:00'

Upvotes: 3

Views: 18002

Answers (2)

leftjoin
leftjoin

Reputation: 38290

unix_timestamp function converts given format to unix timestamp and from_unixtime function converts from unix timestamp to given format:

hive> select from_unixtime(unix_timestamp('01/01/2016 00:00:00','MM/dd/yyyy HH:m:ss'),'yyyy-MM-dd');
OK
2016-01-01
Time taken: 0.118 seconds, Fetched: 1 row(s)

Upvotes: 3

Tim Biegeleisen
Tim Biegeleisen

Reputation: 520908

Can you try using to_date() with the date portion of your timestamp in ISO format:

SELECT 
    cust_id,
    TO_DATE(CONCAT(SUBSTR(activitydate, 7, 4), '-',
                   SUBSTR(activitydate, 0, 2), '-',
                   SUBSTR(activitydate, 4, 2))) activity_date
    type type_of_contact
FROM repl_task
WHERE
    TO_DATE(CONCAT(SUBSTR(activitydate, 7, 4), '-',
                   SUBSTR(activitydate, 0, 2), '-',
                   SUBSTR(activitydate, 4, 2)))
        BETWEEN '2014-01-01' AND '2017-01-01';

If this runs, but is not very performant, then you should consider storing your timestamps in ISO format. Storing dates as text, or as text in a non standard format, carries a penalty with most databases.

Upvotes: 1

Related Questions