CODEWITHSUNDEEP
hadoopsqoopsqoop2
Rj1
Rj1

Reputation: 469

Why sqoop incremental job not saving last updated value in metastore?

I am using sqoop to import data from Oracle to HDFS.

I am importing around 30 tables on daily basis(Incremental).

Today I just found that my sqoop job is not saving last updated value in metastore. It is daily using save column value for incremental import.

Below is the logs of last two days:-

For May 5th 2019- 

19/05/05 01:37:10 INFO tool.ImportTool: Lower bound value: 
TO_TIMESTAMP('2019-04-26 21:25:19.0', 'YYYY-MM-DD HH24:MI:SS.FF')
19/05/05 01:37:10 INFO tool.ImportTool: Upper bound value: 
TO_TIMESTAMP('2019-05-04 21:25:25.0', 'YYYY-MM-DD HH24:MI:SS.FF')

For May 6th 2019- 

19/05/06 01:37:34 INFO tool.ImportTool: Lower bound value: 
TO_TIMESTAMP('2019-04-26 21:25:19.0', 'YYYY-MM-DD HH24:MI:SS.FF')
19/05/06 01:37:34 INFO tool.ImportTool: Upper bound value: 
TO_TIMESTAMP('2019-05-05 21:25:23.0', 'YYYY-MM-DD HH24:MI:SS.FF')

Here you can see its using 2019-04-26 21:25:19.0 value as last updated. It will make duplicacy of data in my destination.

So what should I do to resolve this issue ?

Any help regarding this would be highly appreciated.

Thanks

Upvotes: 1

Views: 85

Answers (0)

Related Questions