CODEWITHSUNDEEP
mavenloggingcassandralog4jlog4j2
Jamie
Jamie

Reputation: 1922

How do I get log4j messages to log to Cassandra?

I’m getting killed here trying to redirect log4j to a Cassandra database. I’ve spent an enormous amount of time trying everything I could think of, so I won’t be able to cover everything I’ve tried, but I’ll try to describe what I’m trying to do and what I’m running into as concisely as I can.

Our codebase currently uses log4j 1.2.17, so my first try is to get com.datastax.logging.appender.CassandraAppender working.
The relevant Maven dependency is this:

<dependency>
    <groupId>com.datastax.logging</groupId>
    <artifactId>cassandra-log4j-appender</artifactId>
    <version>3.1.0</version>
</dependency>

The relevant appender entry in log4j.xml is this: 

<appender name="cassandra" class="com.datastax.logging.appender.CassandraAppender"/>

The appender defaults to “localhost”, or “127.0.0.1”, I don’t remember which, and I’ve tried both explicitly. It also defaults to port 9042, which is what my Cassandra is configured for. In fact, here’s the console output from Cassandra:

Binding thrift service to localhost/127.0.0.1:9160

All I can get out of this is an exception:

log4j:ERROR Error
com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (null))
    at com.datastax.driver.core.ControlConnection.reconnectInternal(ControlConnection.java:196)
    at com.datastax.driver.core.ControlConnection.connect(ControlConnection.java:80)
    at com.datastax.driver.core.Cluster$Manager.init(Cluster.java:1145)
    at com.datastax.driver.core.Cluster.init(Cluster.java:149)
    at com.datastax.driver.core.Cluster.connect(Cluster.java:225)
    at com.datastax.logging.appender.CassandraAppender.initClient(CassandraAppender.java:141)
    at com.datastax.logging.appender.CassandraAppender.append(CassandraAppender.java:97)
    at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
    at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
    at org.apache.log4j.Category.callAppenders(Category.java:206)
    <snip>
log4j:ERROR Error setting up cassandra logging schema: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (null))

After banging my head against the wall, I thought I would try to get the Apache Cassandra appender working. Unfortunately, it’s only for log4j2, and updating our codebase is non-trivial. That said, I’m having even less luck with it.

I created a small test project to try to get something working. In log4j2, Cassandra is supposed to be a built-in appender type, according to https://logging.apache.org/log4j/2.x/manual/appenders.html#CassandraAppender. However, it absolutely can’t find it when I specify it in my log4j2.xml file. Here’s my file:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="INFO">
<Appenders>
    <Console name="Console" target="SYSTEM_OUT">
        <PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n" />
    </Console>
    <File name="MyFile" fileName="all.log" immediateFlush="false" append="false">
        <PatternLayout pattern="%d{yyy-MM-dd HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"/>
    </File>
    <Cassandra name="Cassandra" clusterName="Test Cluster" keyspace="gii" table="gii_event_log" bufferSize="10" batched="true">
        <SocketAddress host="localhost" port="9042"/>
        <ColumnMapping name="id" pattern="%uuid{TIME}" type="java.util.UUID"/>
        <ColumnMapping name="identifier" pattern="%marker"/>
        <ColumnMapping name="message" pattern="%message"/>
        <ColumnMapping name="priority" pattern="%level"/>
        <ColumnMapping name="scope" pattern="%level"/>
        <ColumnMapping name="time_stamp" literal="now()"/>
        <ColumnMapping name="type" pattern="%level"/>
    </Cassandra>
</Appenders>
<Loggers>
    <Root level="debug">
        <AppenderRef ref="Console"/>
        <AppenderRef ref="MyFile"/>
        <AppenderRef ref="Cassandra"/>
    </Root>
</Loggers>
</Configuration>

Here’s what happens:

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/logging/log4j/LogManager
        at com.ge.enconn.TestClass.<init>(TestClass.java:10)
        at com.ge.enconn.App.main(App.java:11)
Caused by: java.lang.ClassNotFoundException: org.apache.logging.log4j.LogManager
        at java.net.URLClassLoader.findClass(Unknown Source)
        at java.lang.ClassLoader.loadClass(Unknown Source)
        at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
        at java.lang.ClassLoader.loadClass(Unknown Source)
        ... 2 more

Here are the relevant dependencies from my pom file:

<dependency>
  <groupId>org.apache.logging.log4j</groupId>
  <artifactId>log4j-api</artifactId>
  <version>${log4j.version}</version>
</dependency>
<dependency>
  <groupId>org.apache.logging.log4j</groupId>
  <artifactId>log4j-core</artifactId>
  <version>${log4j.version}</version>
</dependency>
<dependency>
  <groupId>org.apache.logging.log4j</groupId>
  <artifactId>log4j-nosql</artifactId>
  <version>${log4j.version}</version>
</dependency>
<dependency>
  <groupId>com.datastax.cassandra</groupId>
  <artifactId>cassandra-driver-core</artifactId>
  <version>3.1.0</version>
</dependency>

The pom for log4j-nosql is here: https://github.com/apache/logging-log4j2/blob/master/log4j-nosql/pom.xml. This should get it into my classpath, right?

Upvotes: 5

Views: 2314

Answers (2)

Dimple
Dimple

Reputation: 1

As soon as I added the following dependency the error was resolved.

ERROR: , Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/logging/log4j/LogManager

Dependency:

<dependency>
  <groupId>org.apache.logging.log4j</groupId>
  <artifactId>log4j-cassandra</artifactId>
  <version>2.11.1</version>
</dependency>

Upvotes: 0

Ashraful Islam
Ashraful Islam

Reputation: 12840

First Use this inside dependencies tag of the pom file : 

<!-- Apache Cassandra -->
<dependency>
    <groupId>com.datastax.cassandra</groupId>
    <artifactId>cassandra-driver-core</artifactId>
    <version>3.1.4</version>
</dependency>

<!-- Log4j2 -->
<dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-api</artifactId>
    <version>2.8.1</version>
</dependency>
<dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-core</artifactId>
    <version>2.8.1</version>
</dependency>
<dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-1.2-api</artifactId>
    <version>2.8.1</version>
</dependency>
<dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-jcl</artifactId>
    <version>2.8.1</version>
</dependency>
<dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-slf4j-impl</artifactId>
    <version>2.8.1</version>
</dependency>

<dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-nosql</artifactId>
    <version>2.8.1</version>
</dependency>

Second Create table in Cassandra : 

CREATE TABLE gii_event_log (
    id timeuuid PRIMARY KEY,
    identifier text,
    message text,
    priority text,
    scope text,
    time_stamp timeuuid,
    type text
);

Third Update your log4j2 file check host value if you get No Host available ERROR. And add username and password inside Cassandra tag if you get Authentication ERROR.

Sample Java Code : 

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;

/**
 *
 * @author Ashraful Islam
 */
public class CassandraLog {

    private final static Logger LOGGER = LogManager.getLogger();

    public void printDemo() {
        for (int i = 0; i < 10; i++) {
            System.out.println(i);
            LOGGER.info("Testing {}", i);
            LOGGER.error("Testing {}", i);
            LOGGER.debug("Testing {}", i);
        }
    }

    /**
     * @param args the command line arguments
     */
    public static void main(String[] args) {
        new CassandraLog().printDemo();
    }

}

Output : 

id                                   | identifier | message   | priority | scope | time_stamp                           | type
--------------------------------------+------------+-----------+----------+-------+--------------------------------------+-------
 b85caa03-0ecb-11e7-af5e-b083fe92c73a |            | Testing 1 |    ERROR | ERROR | b85222b2-0ecb-11e7-a374-55d83eefb705 | ERROR
 b85eccec-0ecb-11e7-af5e-b083fe92c73a |            | Testing 4 |    DEBUG | DEBUG | b85222bb-0ecb-11e7-a374-55d83eefb705 | DEBUG
 b8602c83-0ecb-11e7-af5e-b083fe92c73a |            | Testing 6 |    ERROR | ERROR | b85222c2-0ecb-11e7-a374-55d83eefb705 | ERROR
 b862285b-0ecb-11e7-af5e-b083fe92c73a |            | Testing 9 |    DEBUG | DEBUG | b85222ca-0ecb-11e7-a374-55d83eefb705 | DEBUG

Upvotes: 4

Related Questions