Shubham
Reputation: 21
How to solve the classNotFoundException warnings when running Kafka Connect on windows?
I have checked multiple topics on the StackOverflow and Kafka Jira board and found that these warnings can be suppressed from connect-log4j.properperties file by setting reflections to log at error level. But how to solve these warnings completely.
C:\xxx\apache-kafka\kafka_2.13-2.4.0\bin\windows>connect-distributed.bat C:\xxx\apache-kafka\kafka_2.13-2.4.0\config\connect-distributed.properties
log4j:ERROR setFile(null,true) call failed.
java.io.FileNotFoundException: C:\xxx\apache-kafka\kafka_2.13-2.4.0\logs\connect.log (Access is denied)
at java.io.FileOutputStream.open0(Native Method)
at java.io.FileOutputStream.open(FileOutputStream.java:270)
at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
at java.io.FileOutputStream.<init>(FileOutputStream.java:133)
at org.apache.log4j.FileAppender.setFile(FileAppender.java:294)
at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:165)
at org.apache.log4j.DailyRollingFileAppender.activateOptions(DailyRollingFileAppender.java:223)
at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)
at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104)
at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:842)
at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768)
at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580)
at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526)
at org.apache.log4j.LogManager.<clinit>(LogManager.java:127)
at org.slf4j.impl.Log4jLoggerFactory.<init>(Log4jLoggerFactory.java:66)
at org.slf4j.impl.StaticLoggerBinder.<init>(StaticLoggerBinder.java:72)
at org.slf4j.impl.StaticLoggerBinder.<clinit>(StaticLoggerBinder.java:45)
at org.slf4j.LoggerFactory.bind(LoggerFactory.java:150)
at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:124)
at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:412)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:357)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:383)
at org.apache.kafka.connect.cli.ConnectDistributed.<clinit>(ConnectDistributed.java:57)
log4j:ERROR Either File or DatePattern options are not set for appender [connectAppender].
[2020-01-24 13:26:44,426] INFO WorkerInfo values:
jvm.args = -Xmx256M, -XX:+UseG1GC, -XX:MaxGCPauseMillis=20, -XX:InitiatingHeapOccupancyPercent=35, -XX:+ExplicitGCInvokesConcurrent, -Djava.awt.headless=true, -Dcom.sun.management.jmxremote, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.ssl=false, -Dkafka.logs.dir=C:\xxx\apache-kafka\kafka_2.13-2.4.0/logs, -Dlog4j.configuration=file:C:\xxx\apache-kafka\kafka_2.13-2.4.0/config/connect-log4j.properties
jvm.spec = , OpenJDK 64-Bit Server VM, 1.8.0_222-2-ojdkbuild, 25.222-b10
jvm.classpath = C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\activation-1.1.1.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\aopalliance-repackaged-2.5.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\argparse4j-0.7.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\audience-annotations-0.5.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\commons-cli-1.4.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\commons-lang3-3.8.1.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\connect-api-2.4.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\connect-basic-auth-extension-2.4.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\connect-file-2.4.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\connect-json-2.4.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\connect-mirror-2.4.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\connect-mirror-client-2.4.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\connect-runtime-2.4.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\connec
t-transforms-2.4.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\guava-20.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\hk2-api-2.5.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\hk2-locator-2.5.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\hk2-utils-2.5.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\jackson-annotations-2.10.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\jackson-core-2.10.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\jackson-databind-2.10.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\jackson-dataformat-csv-2.10.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\jackson-datatype-jdk8-2.10.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\jackson-jaxrs-base-2.10.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\jackson-jaxrs-json-provider-2.10.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\jackson-module-jaxb-annotations-2.10.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\jackson-module-paranamer-2.10.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\jackson-module-s
cala_2.13-2.10.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\jakarta.activation-api-1.2.1.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\jakarta.annotation-api-1.3.4.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\jakarta.inject-2.5.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\jakarta.ws.rs-api-2.1.5.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\jakarta.xml.bind-api-2.3.2.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\javassist-3.22.0-CR2.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\javax.servlet-api-3.1.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\javax.ws.rs-api-2.1.1.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\jaxb-api-2.3.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\jersey-client-2.28.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\jersey-common-2.28.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\jersey-container-servlet-2.28.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\jersey-container-servlet-core-2.28.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\jersey-hk2-2.28.jar;C:\u
hx\apache-kafka\kafka_2.13-2.4.0\libs\jersey-media-jaxb-2.28.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\jersey-server-2.28.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\jetty-client-9.4.20.v20190813.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\jetty-continuation-9.4.20.v20190813.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\jetty-http-9.4.20.v20190813.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\jetty-io-9.4.20.v20190813.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\jetty-security-9.4.20.v20190813.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\jetty-server-9.4.20.v20190813.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\jetty-servlet-9.4.20.v20190813.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\jetty-servlets-9.4.20.v20190813.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\jetty-util-9.4.20.v20190813.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\jopt-simple-5.0.4.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\kafka-clients-2.4.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\kafka-log4j
-appender-2.4.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\kafka-streams-2.4.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\kafka-streams-examples-2.4.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\kafka-streams-scala_2.13-2.4.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\kafka-streams-test-utils-2.4.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\kafka-tools-2.4.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\kafka_2.13-2.4.0-javadoc.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\kafka_2.13-2.4.0-javadoc.jar.asc;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\kafka_2.13-2.4.0-scaladoc.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\kafka_2.13-2.4.0-scaladoc.jar.asc;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\kafka_2.13-2.4.0-sources.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\kafka_2.13-2.4.0-sources.jar.asc;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\kafka_2.13-2.4.0-test-sources.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\kafka_2.13-2.4.0-test-sources.jar.asc;C:\xxx\apache-kafka\kafka
_2.13-2.4.0\libs\kafka_2.13-2.4.0-test.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\kafka_2.13-2.4.0-test.jar.asc;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\kafka_2.13-2.4.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\kafka_2.13-2.4.0.jar.asc;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\log4j-1.2.17.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\lz4-java-1.6.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\maven-artifact-3.6.1.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\metrics-core-2.2.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\netty-buffer-4.1.42.Final.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\netty-codec-4.1.42.Final.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\netty-common-4.1.42.Final.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\netty-handler-4.1.42.Final.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\netty-resolver-4.1.42.Final.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\netty-transport-4.1.42.Final.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\netty-transport-native-e
poll-4.1.42.Final.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\netty-transport-native-unix-common-4.1.42.Final.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\osgi-resource-locator-1.0.1.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\paranamer-2.8.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\plexus-utils-3.2.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\reflections-0.9.11.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\rocksdbjni-5.18.3.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\scala-collection-compat_2.13-2.1.2.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\scala-java8-compat_2.13-0.9.0.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\scala-library-2.13.1.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\scala-logging_2.13-3.9.2.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\scala-reflect-2.13.1.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\slf4j-api-1.7.28.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\slf4j-log4j12-1.7.28.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\snappy-java-1.1.7.3.ja
r;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\validation-api-2.0.1.Final.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\zookeeper-3.5.6.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\zookeeper-jute-3.5.6.jar;C:\xxx\apache-kafka\kafka_2.13-2.4.0\libs\zstd-jni-1.4.3-1.jar
os.spec = Windows Server 2012 R2, amd64, 6.3
os.vcpus = 1
(org.apache.kafka.connect.runtime.WorkerInfo:71)
[2020-01-24 13:26:44,468] INFO Scanning for plugin classes. This might take a moment ... (org.apache.kafka.connect.cli.ConnectDistributed:90)
[2020-01-24 13:26:44,588] INFO Loading plugin from: C:\xxx\apache-kafka\kafka_2.13-2.4.0\plugins\kafka-connect-redis-0.0.2-SNAPSHOT-jar-with-dependencies.jar (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:241)
[2020-01-24 13:26:50,288] INFO Reflections took 5556 ms to scan 1 urls, producing 1898 keys and 8262 values [using 1 cores] (org.reflections.Reflections:232)
[2020-01-24 13:26:50,386] WARN could not get type for name io.lettuce.core.dynamic.support.TypeInformation from any class loader (org.reflections.Reflections:396)
org.reflections.ReflectionsException: could not get type for name io.lettuce.core.dynamic.support.TypeInformation
at org.reflections.ReflectionUtils.forName(ReflectionUtils.java:390)
at org.reflections.Reflections.expandSuperTypes(Reflections.java:381)
at org.reflections.Reflections.<init>(Reflections.java:126)
at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader$InternalReflections.<init>(DelegatingClassLoader.java:428)
at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanPluginPath(DelegatingClassLoader.java:327)
at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanUrlsAndAddPlugins(DelegatingClassLoader.java:263)
at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.registerPlugin(DelegatingClassLoader.java:255)
at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.initPluginLoader(DelegatingClassLoader.java:224)
at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.initLoaders(DelegatingClassLoader.java:201)
at org.apache.kafka.connect.runtime.isolation.Plugins.<init>(Plugins.java:60)
at org.apache.kafka.connect.cli.ConnectDistributed.startConnect(ConnectDistributed.java:91)
at org.apache.kafka.connect.cli.ConnectDistributed.main(ConnectDistributed.java:78)
Caused by: java.lang.ClassNotFoundException: io.lettuce.core.dynamic.support.TypeInformation
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.reflections.ReflectionUtils.forName(ReflectionUtils.java:388)
... 11 more
[2020-01-24 13:26:50,396] WARN could not get type for name org.jdom.output.XMLOutputter from any class loader (org.reflections.Reflections:396)
org.reflections.ReflectionsException: could not get type for name org.jdom.output.XMLOutputter
at org.reflections.ReflectionUtils.forName(ReflectionUtils.java:390)
at org.reflections.Reflections.expandSuperTypes(Reflections.java:381)
at org.reflections.Reflections.<init>(Reflections.java:126)
at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader$InternalReflections.<init>(DelegatingClassLoader.java:428)
at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanPluginPath(DelegatingClassLoader.java:327)
at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanUrlsAndAddPlugins(DelegatingClassLoader.java:263)
at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.registerPlugin(DelegatingClassLoader.java:255)
at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.initPluginLoader(DelegatingClassLoader.java:224)
at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.initLoaders(DelegatingClassLoader.java:201)
at org.apache.kafka.connect.runtime.isolation.Plugins.<init>(Plugins.java:60)
at org.apache.kafka.connect.cli.ConnectDistributed.startConnect(ConnectDistributed.java:91)
at org.apache.kafka.connect.cli.ConnectDistributed.main(ConnectDistributed.java:78)
. . . . same logs . . . . .
---------------------------
Caused by: java.lang.ClassNotFoundException: org.jdom.output.XMLOutputter
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.reflections.ReflectionUtils.forName(ReflectionUtils.java:388)
... 11 more
Terminate batch job (Y/N)? n
C:\xxx\apache-kafka\kafka_2.13-2.4.0\bin\windows>
Connector configurations distributed mode:
{
"name": "RedisSinkConnector1",
"config": {
"name": "RedisSinkConnector1",
"connector.class": "com.github.jcustenborder.kafka.connect.redis.RedisSinkConnector",
"tasks.max": 3,
"topics": "cache_record",
"redis.hosts": "XXX.XXX.X.XXX:XXXX",
"redis.password": "XXXXXXX",
"schemas.enable": "false",
"value.converter.schemas.enable": "false",
"key.converter": "org.apache.kafka.connect.storage.StringConverter",
"value.converter": "org.apache.kafka.connect.storage.StringConverter",
"errors.tolerance" : "all",
"errors.deadletterqueue.topic.name" :"fail_record",
"errors.deadletterqueue.context.headers.enable" : true,
"errors.log.enable" : true,
"errors.log.include.messages" : true,
}
}
Kafka version: kafka_2.13-2.4.0 Connector: https://www.confluent.io/hub/jcustenborder/kafka-connect-redis OS: Windows Server
Upvotes: 0
Views: 1838
Answers (1)
OneCricketeer
Reputation: 191758
There are several open pull requests in Kafka that fix quoting issues around forming the CLASSPATH in the Kafka start scripts. Mainly, I think the problem is paths with spaces breaks the script
That being said, you could try setting the CLASSPATH variable yourself to include the path of the folder containing the kafka libraries, or you can run Connect in a supported Linux environment such as a VM, WSL/Cygwin, or Docker
Upvotes: 2
Related Questions
- Failed to find any class that implements connector and which name matches FileStreamSource
- Caused by: java.lang.ClassNotFoundException: kafka.admin.TopicCommand
- Kafka Connect Error : java.lang.NoClassDefFoundError: org/apache/http/conn/HttpClientConnectionManager
- ERROR Stopping due to error (org.apache.kafka.connect.cli.ConnectStandalone) java.lang.NoClassDefFoundError: io/debezium/util/IoUtil
- Kafka installation - classpath is empty
- Apache kafka Setup on windows getting error #
- Not able to run a class using kafka-run-class.bat on windows
- Error While Running Kafka Server on Windows
- Java Kafka Consumer noClassDefFoundError
- kafka startup error ubuntu class load failure