CODEWITHSUNDEEP
apache-kafkaapache-stormworker
M.SH
M.SH

Reputation: 66

Apache Storm is not running properly and worker.log generating exceptions

I am using Apache storm with Apache Kafka. The spout used is kafka-spout, but it is not reading anything from the Kafka topic. I don't know if it is related to the problem I am facing in my storm's workers or not. Please let me know and inform me if you know any relevant solution.

The Storm topology is uploaded successfully to Storm. I fetched the Logs and here is what the workers are writing in their worker.log file:

java.lang.ExceptionInInitializerError
    at java.nio.file.FileSystems.getDefault(FileSystems.java:176) ~[?:1.8.0_91]
    at java.nio.file.Paths.get(Paths.java:84) ~[?:1.8.0_91]
    at org.apache.storm.util$is_absolute_path_QMARK_.invoke(util.clj:68) ~[storm-core-1.0.1.jar:1.0.1]
    at org.apache.storm.config$absolute_storm_local_dir.invoke(config.clj:94) ~[storm-core-1.0.1.jar:1.0.1]
    at org.apache.storm.config$worker_root.invoke(config.clj:290) ~[storm-core-1.0.1.jar:1.0.1]
    at org.apache.storm.config$worker_root.invoke(config.clj:292) ~[storm-core-1.0.1.jar:1.0.1]
    at org.apache.storm.config$worker_pids_root.invoke(config.clj:296) ~[storm-core-1.0.1.jar:1.0.1]
    at org.apache.storm.config$worker_pid_path.invoke(config.clj:304) ~[storm-core-1.0.1.jar:1.0.1]
    at org.apache.storm.daemon.worker$fn__8450$exec_fn__2461__auto____8451.invoke(worker.clj:592) ~[storm-core-1.0.1.jar:1.0.1]
    at clojure.lang.AFn.applyToHelper(AFn.java:178) ~[clojure-1.7.0.jar:?]
    at clojure.lang.AFn.applyTo(AFn.java:144) ~[clojure-1.7.0.jar:?]
    at clojure.core$apply.invoke(core.clj:630) ~[clojure-1.7.0.jar:?]
    at org.apache.storm.daemon.worker$fn__8450$mk_worker__8545.doInvoke(worker.clj:583) [storm-core-1.0.1.jar:1.0.1]
    at clojure.lang.RestFn.invoke(RestFn.java:512) [clojure-1.7.0.jar:?]
    at org.apache.storm.daemon.worker$_main.invoke(worker.clj:771) [storm-core-1.0.1.jar:1.0.1]
    at clojure.lang.AFn.applyToHelper(AFn.java:165) [clojure-1.7.0.jar:?]
    at clojure.lang.AFn.applyTo(AFn.java:144) [clojure-1.7.0.jar:?]
    at org.apache.storm.daemon.worker.main(Unknown Source) [storm-core-1.0.1.jar:1.0.1]

Caused by: java.security.PrivilegedActionException
    at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_91]
    at java.nio.file.FileSystems$DefaultFileSystemHolder.defaultFileSystem(FileSystems.java:96) ~[?:1.8.0_91]
    at java.nio.file.FileSystems$DefaultFileSystemHolder.<clinit>(FileSystems.java:90) ~[?:1.8.0_91]
    ... 18 more

Caused by: sun.nio.fs.UnixException: No such file or directory
    at sun.nio.fs.UnixNativeDispatcher.getcwd(Native Method) ~[?:1.8.0_91]
    at sun.nio.fs.UnixFileSystem.<init>(UnixFileSystem.java:67) ~[?:1.8.0_91]
    at sun.nio.fs.LinuxFileSystem.<init>(LinuxFileSystem.java:39) ~[?:1.8.0_91]
    at sun.nio.fs.LinuxFileSystemProvider.newFileSystem(LinuxFileSystemProvider.java:46) ~[?:1.8.0_91]
    at sun.nio.fs.LinuxFileSystemProvider.newFileSystem(LinuxFileSystemProvider.java:39) ~[?:1.8.0_91]
    at sun.nio.fs.UnixFileSystemProvider.<init>(UnixFileSystemProvider.java:56) ~[?:1.8.0_91]
    at sun.nio.fs.LinuxFileSystemProvider.<init>(LinuxFileSystemProvider.java:41) ~[?:1.8.0_91]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_91]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_91]
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_91]
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_91]
    at java.lang.Class.newInstance(Class.java:442) ~[?:1.8.0_91]
    at sun.nio.fs.DefaultFileSystemProvider.createProvider(DefaultFileSystemProvider.java:48) ~[?:1.8.0_91]
    at sun.nio.fs.DefaultFileSystemProvider.create(DefaultFileSystemProvider.java:63) ~[?:1.8.0_91]
    at java.nio.file.FileSystems$DefaultFileSystemHolder.getDefaultProvider(FileSystems.java:108) ~[?:1.8.0_91]
    at java.nio.file.FileSystems$DefaultFileSystemHolder.access$000(FileSystems.java:89) ~[?:1.8.0_91]
    at java.nio.file.FileSystems$DefaultFileSystemHolder$1.run(FileSystems.java:98) ~[?:1.8.0_91]
    at java.nio.file.FileSystems$DefaultFileSystemHolder$1.run(FileSystems.java:96) ~[?:1.8.0_91]
    at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_91]
    at java.nio.file.FileSystems$DefaultFileSystemHolder.defaultFileSystem(FileSystems.java:96) ~[?:1.8.0_91]
    at java.nio.file.FileSystems$DefaultFileSystemHolder.<clinit>(FileSystems.java:90) ~[?:1.8.0_91]
    ... 18 more

2016-07-24 13:14:33.154 o.a.s.util [ERROR] Halting process: ("Error on initialization")
java.lang.RuntimeException: ("Error on initialization")
    at org.apache.storm.util$exit_process_BANG_.doInvoke(util.clj:341) [storm-core-1.0.1.jar:1.0.1]
    at clojure.lang.RestFn.invoke(RestFn.java:423) [clojure-1.7.0.jar:?]
    at org.apache.storm.daemon.worker$fn__8450$mk_worker__8545.doInvoke(worker.clj:583) [storm-core-1.0.1.jar:1.0.1]
    at clojure.lang.RestFn.invoke(RestFn.java:512) [clojure-1.7.0.jar:?]
    at org.apache.storm.daemon.worker$_main.invoke(worker.clj:771) [storm-core-1.0.1.jar:1.0.1]
    at clojure.lang.AFn.applyToHelper(AFn.java:165) [clojure-1.7.0.jar:?]
    at clojure.lang.AFn.applyTo(AFn.java:144) [clojure-1.7.0.jar:?]
    at org.apache.storm.daemon.worker.main(Unknown Source) [storm-core-1.0.1.jar:1.0.1]

Upvotes: 1

Views: 829

Answers (1)

M.SH
M.SH

Reputation: 66

The solution was to check the number of task parallelism in the local cluster of Storm. In their latest version, they added a maximum number for this task parallelism property.That was automatically deleting the folder of the extra worker. It was just changing the number of workers from 3 to 2 or less to execute normally.

int workers = 2;

conf.setMaxTaskParallelism(workers);

Thanks all for your comments.

Upvotes: 1

Related Questions