CODEWITHSUNDEEP
hadoopimporthdfssqoop
DAVID_ROA
DAVID_ROA

Reputation: 309

Sqoop stucks at 5% of progress

I am using Sqoop for importing data from oracle to HDFS. When Job starts it stucks in 5% of progress for about 1 hours and this info is outputs:

INFO mapreduce.Job: Task Id : attempt_1535519556038_0015_m_000037_0, Status : FAILED
Container launch failed for container_1535519556038_0015_01_000043 : org.apache.hadoop.yarn.exceptions.YarnException: Unauthorized request to start container.
This token is expired. current time is 1536133107764 found 1536133094775
Note: System times on machines may be out of sync. Check system time and time zones.
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.instantiateException(SerializedExceptionPBImpl.java:168)
        at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.deSerialize(SerializedExceptionPBImpl.java:106)
        at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$Container.launch(ContainerLauncherImpl.java:155)
        at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$EventProcessor.run(ContainerLauncherImpl.java:375)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

and then it continues until the jobs successfully terminate and all the data imported. So, My question is What is the reason for hanging the job in 5% of progress? Why is it self-correcting? Is it normal? If not, Is it possible to relate to that issued info? How can I fix that?

Upvotes: 0

Views: 158

Answers (1)

CHEEKATLAPRADEEP
CHEEKATLAPRADEEP

Reputation: 12768

The error message clearly explains “Unauthorized request to start container. This token is expired”.

One of the options would be increasing lifespan of container by setting: yarn.resourcemanager.rm.container-allocation.expiry-interval-ms which is by default is 10 minutes.

Note: The jobs will work if you increase the yarn.resourcemanager.rm.container-allocation.expiry-interval-ms in the yarn-site.xml config file.

<property>
   <name>yarn.resourcemanager.rm.container-allocation.expiry-interval-ms</name>
   <value>1000000</value>
</property>

Upvotes: 1

Related Questions