CODEWITHSUNDEEP
hadoophiveapache-tez
arjun jain
arjun jain

Reputation: 11

Apache Tez Job fails due to java.lang.NumberFormatException for input string: "30s"

I am trying to execute query on Apache hive on tez but somehow i am getting the error as below and I have no clue how to solve it

  1. Apache Hadoop 3.1.1
  2. Apache Hive 3.1.0
  3. Apache Tez 0.9.1

My tez-site.xml


    <configuration>
        <property>
            <name>tez.lib.uris</name>
            <value>hdfs://localhost:8020/apps/apache-tez-0.9.1-bin/share/tez.tar.gz</value>
        </property>
        <property>
            <name>tez.staging-dir</name>
            <value>/tmp/${user.name}/staging</value>
        </property>
    <configuration>


020-04-22 21:08:55,530 [INFO] [main] |shim.HadoopShimsLoader|: Trying to locate HadoopShimProvider for hadoopVersion=2.7.0, majorVersion=2, minorVersion=7
2020-04-22 21:08:55,531 [INFO] [main] |shim.HadoopShimsLoader|: Picked HadoopShim org.apache.tez.hadoop.shim.HadoopShim27, providerName=org.apache.tez.hadoop.shim.HadoopShim25_26_27Provider, overrideProviderViaConfig=null, hadoopVersion=2.7.0, majorVersion=2, minorVersion=7
2020-04-22 21:08:55,551 [INFO] [main] |app.DAGAppMaster|: AM Level configured TaskSchedulers: [0:TezYarn:null],[1:TezUber:null]
2020-04-22 21:08:55,551 [INFO] [main] |app.DAGAppMaster|: AM Level configured ContainerLaunchers: [0:TezYarn:null],[1:TezUber:null]
2020-04-22 21:08:55,551 [INFO] [main] |app.DAGAppMaster|: AM Level configured TaskCommunicators: [0:TezYarn:null],[1:TezUber:null]
2020-04-22 21:08:55,551 [INFO] [main] |app.DAGAppMaster|: Comparing client version with AM version, clientVersion=0.9.1, AMVersion=0.9.1
2020-04-22 21:08:55,633 [INFO] [main] |service.AbstractService|: Service org.apache.tez.dag.app.DAGAppMaster failed in state INITED; cause: java.lang.NumberFormatException: For input string: "30s"
java.lang.NumberFormatException: For input string: "30s"
    at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
    at java.lang.Long.parseLong(Long.java:589)
    at java.lang.Long.parseLong(Long.java:631)
    at org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1311)
    at org.apache.hadoop.hdfs.DFSClient$Conf.<init>(DFSClient.java:502)
    at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:637)
    at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:619)
    at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149)
    at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2653)
    at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:92)
    at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2687)
    at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2669)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371)
    at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
    at org.apache.tez.common.TezCommonUtils.getTezBaseStagingPath(TezCommonUtils.java:87)
    at org.apache.tez.common.TezCommonUtils.getTezSystemStagingPath(TezCommonUtils.java:146)
    at org.apache.tez.dag.app.DAGAppMaster.serviceInit(DAGAppMaster.java:492)
    at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
    at org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2662)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
    at org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659)
    at org.apache.tez.dag.app.DAGAppMaster.main(DAGAppMaster.java:2464)
2020-04-22 21:08:55,636 [WARN] [main] |service.AbstractService|: When stopping the service org.apache.tez.dag.app.DAGAppMaster : java.lang.NullPointerException
java.lang.NullPointerException

Upvotes: 1

Views: 2415

Answers (3)

alen
alen

Reputation: 41

try to add this property in hdfs-site.xml

<property>
    <name>dfs.client.datanode-restart.timeout</name>
    <value>30</value>
</property>

Upvotes: 4

Diana cdmx
Diana cdmx

Reputation: 11

The default value of the property dfs.client.datanode-restart.timeout is 30s. This issue is related.

Here is mentioned a workaround. It worked for me.

Upvotes: 1

arjun jain
arjun jain

Reputation: 11

Thanks for you reply.. i have already checked my hdfs-site.xml there is no setting that indicate that have value specified as "30s".

<property>
        <name>dfs.replication</name>
        <value>1</value>
</property>
<property>
    <name>dfs.namenode.name.dir</name>
    <value>file:///hadoopdata/hdfs/namenode</value>
</property>
<property>
    <name>dfs.datanode.data.dir</name>
    <value>file:///hadoopdata/hdfs/datanode</value>
</property>
<property>
    <name>dfs.blocksize</name>
    <value>268435456</value>
</property>
<property>
    <name>dfs.blocksize</name>
    <value>268435456</value>
</property>
<property>
    <name>dfs.namenode.handler.count</name>
    <value>100</value>
</property>
<property>
    <name>dfs.permissions.superusergroup</name>
    <value>hadoop</value>
    <description>The name of the group of super-users.</description>
</property>

Upvotes: 0

Related Questions