CODEWITHSUNDEEP
jbosslogbacklogstashwildfly
user3604590
user3604590

Reputation: 11

JBoss/WildFly Logstash

I've been having trouble getting our Wildfly logs onto our Logstash/Kibana server Wildfly is on an Ubuntu AWS server as well as logstash/kibana

/etc/opt/logstash.conf 

input {
  file {
    type => "webapplog"
    path => "/home/ubuntu/app/log/logstash.json"
    codec => "json"
    tags => [ "tariff-engine" ]
  }
}

output {
  redis { 
    host => "redis.mvp.enernoc.net"
    data_type => "list"
    batch => true
    congestion_threshold => 1000
    key => "logstash"
  }
}

/home/ubuntu/app/config/logback.xml

<?xml version="1.0" encoding="UTF-8"?>
<configuration>

    <appender name="logstash" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
            <level>INFO</level>
        </filter>
        <file>/home/ubuntu/app/log/logstash.json</file>
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <fileNamePattern>/home/ubuntu/app/log/logstash.json.%d{yyyy-MM-dd}            </fileNamePattern>
            <maxHistory>7</maxHistory>
        </rollingPolicy>
    <encoder class="net.logstash.logback.encoder.LogstashEncoder" />
    </appender>

    <root level="INFO">
        <appender-ref ref="logstash" />
    </root>

</configuration>

/etc/opt/init.d/jboss

#!/bin/bash
### BEGIN INIT INFO
# Provides:          jboss
# Required-Start:    $local_fs $remote_fs $network $syslog
# Required-Stop:     $local_fs $remote_fs $network $syslog
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Start/Stop JBoss AS v7.0.0
### END INIT INFO

export PID_FILE=/home/ubuntu/app/pid/jboss.pid
export LOG_FILE=/home/ubuntu/app/log/jboss.log

export JAVA_HOME=/usr/java/default

#define where jboss is - this is the directory containing directories log, bin, conf etc
export JBOSS_HOME=/usr/local/jboss/default

#define the user under which jboss will run, or use 'RUNASIS' to run as the current user
export JBOSS_USER=jboss

#make sure java is in your path
JAVAPTH=${JAVAPTH:-"$JAVA_HOME/bin/java"}
export PATH=$PATH:$JAVAPTH

#source some script files in order to set and export environmental
#variables
#as well as add the appropriate executables to $PATH
[ -r /etc/profile.d/java.sh ] && . /etc/profile.d/java.sh
[ -r /etc/profile.d/jboss.sh ] && . /etc/profile.d/jboss.sh

start_up() {
    echo "Starting JBoss Application Server"
    sudo -u $JBOSS_USER nohup ${JBOSS_HOME}/bin/standalone.sh -b 0.0.0.0 -bmanagement     0.0.0.0 -DtariffEngine.URLsPropertyFile='file:///opt/enernoc/resource/tariff-engine-    urls.properties' -DtariffEngine.PropertiesFile='file:///opt/enernoc/resource/te-    properties.properties' -Dlogback.configurationFile='/home/ubuntu/app/config/logback.xml'  >>     $LOG_FILE 2>&1 &
    echo $! > $PID_FILE
}

shut_down() {
    kill `cat $PID_FILE` && rm $PID_FILE && echo "Killed JBoss" || echo "JBoss not     running"
}


case "$1" in
    start)
        start_up
    ;;
    stop)
    shut_down
    ;;
    restart)
        shut_down
        sleep 1
        start_up        
    ;;
    *)
        echo "Usage: /etc/init.d/jboss {start|stop|restart}"
        exit 1
    ;;
esac

exit 0

I have tried using jboss-deployment-structure.xml having excludes on logging subsystem, with no evail

also note that the jboss.log in /home/ubuntu/app/app contains all logging, while logstash.json is empty

Upvotes: 1

Views: 3513

Answers (2)

mtpettyp
mtpettyp

Reputation: 5569

I prefer to use the built-in WildFly logging in order to accomplish this.

SYNAXON/logstash-util-formatter can easily be used as a WildFly custom-formatter to output it's logs in logstash's JSON format. Note that this project (currently) hasn't been published to Maven Central so you'll have to built the JAR yourself.

Installation Steps:

  1. Build SYNAXON/logstash-util-formatter
  2. Copy logstash-util-formatter-1.0.jar to [wildfly-dir]/modules/system/layers/base/net/logstash/main

  3. Create [wildfly-dir]/modules/system/layers/base/net/logstash/main/module.xml:

    <?xml version="1.0" encoding="UTF-8"?>
<module xmlns="urn:jboss:module:1.1" name="net.logstash">
    <resources>
        <resource-root path="logstash-util-formatter-1.0.jar" />
    </resources>
    <dependencies>
        <module name="javax.json.api"/>
    </dependencies>
</module>

  4. Update your standalone.xml:

    ....
<!-- Could also be file-handler or size-rotating-file-handler -->
<periodic-rotating-file-handler name="LOGSTASH"
        autoflush="true">
    <level name="INFO" />
    <formatter>
        <named-formatter name="LOGSTASH-PATTERN" />
    </formatter>
    <file relative-to="jboss.server.log.dir" path="logstash.log" />
    <suffix value=".yyyy-MM-dd" />
    <append value="true" />
</periodic-rotating-file-handler>
....
<root-logger>
    <level name="INFO"/>
    <handlers>
        <handler name="CONSOLE"/>
        <handler name="FILE"/>
        <handler name="LOGSTASH" />
    </handlers>
</root-logger>
....
<formatter name="LOGSTASH-PATTERN">
    <custom-formatter class="net.logstash.logging.formatter.LogstashUtilFormatter" module="net.logstash"/>
</formatter>
....

Upvotes: 0

mp911de
mp911de

Reputation: 18119

You could also use native GELF instead taking the detour over a file. logstash provides a GELF input

Logstash input

input { gelf {} }}

which accepts an UDP data stream. The GELF library https://github.com/mp911de/logstash-gelf has connectors for logback and a connector for JBoss/WildFly. Logs of your JBoss and your application can be sent over the same channels so you do not need to parse logfiles anymore.

Logback configuration

<configuration>
   ...
    <appender name="gelf" class="biz.paluch.logging.gelf.logback.GelfLogbackAppender">
        <host>udp:localhost</host>
        <port>12201</port>
        ...
        <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
            <level>INFO</level>
        </filter>
    </appender>
    ...
</configuration>

Upvotes: 1

Related Questions