NikolaS
Reputation: 523
How to log HTTP requests in file in Dropwizard?
I'm having issue in my Dropwizard (version 1.3.8) app logging HTTP Requests into file. I've followed Dropwizard's documentation for configuration of YAML file and my config.yml looks like this:
logging:
level: INFO
loggers:
com.nikolas.master_thesis: DEBUG
appenders:
- type: file
currentLogFilename: /home/nikola/Documents/DWApp-LOGs/DW-Bookshop_LOG.log
threshold: ALL
queueSize: 512
archive: true
archivedLogFilenamePattern: /home/nikola/Documents/DWApp-LOGs/DW-Bookshop_LOG-%d.log
archivedFileCount: 5
timeZone: UTC
logFormat: "%-5p [%d{ISO8601,UTC}] %c: %m%n%rEx"
# logFormat: "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\"" combined
# logFormat: “%h %l %u %t “%r“ %s %b “%{Referer}i“ “%{User-Agent}i““ combined
# logFormat: '%h %l %u %t '%r' %\\>s %b '%{Referer}i' '%{User-Agent}i''
# logFormat: "%d{HH:mm:ss.SSS} [%thread] %-5level %com.nikolas.master_thesis.examples.`config-kt`.logger{36} - %msg%n"
# logFormat: “%h %l %u %t “%r“ %s %b “%{Referer}i“ “%{User-Agent}i““
# logFormat: "%h %l %u %t \"%r\" %s %O \"%{Referer}i\""
# logFormat: “%h %l %u %t “%r“ %\\>s %b “%{Referer}i“ “%{User-Agent}i““
# logFormat: "%h %l %u %t \"%r\" \"%>s\" %O \"%{Referer}i\""
# logFormat: "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
# logFormat: combined
# logFormat: "%h %l %u %t \"%r\" %>s %b" /etc/httpd
# logFormat: “%h %l %u %t “%r” %s %b “%{User-Agent}i””
# logFormat: "%h %l %u %t \"%r\" %>s %b" /etc/httpd
# logFormat: "%h %l %u %t \"%r\" %>s %b"
# logFormat: "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\""
bufferSize: 8KB
immediateFlush: true
# filterFactories:
# - type: URI
- type: console
threshold: ALL
queueSize: 512
discardingThreshold: 0
timeZone: UTC
target: stdout
logFormat: "%-5p [%d{ISO8601,UTC}] %c: %m%n%rEx"
The thing is that HTTP Request is logged in console like this:
192.168.1.5 - - [09/Aug/2020:10:08:05 +0000] "GET /api/orders HTTP/1.1" 200 14094 "-" "Apache-HttpClient/4.5.10 (Java/1.8.0_161)" 710
...and I want to log it also in file, but it does NOT log it at all! I've tried to use commented logFormats, but I get error for each of those commented logFormat values.
E.g. for logFormat: "%h %l %u %t \"%r\" %>s %b" when I run it in terminal (I'm using RMI parameters to collect JMX samples from remote machine):
java -Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.port=1616 -Dcom.sun.management.jmxremote.rmi.port=1616 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -jar ./target/DW-Bookshop-1.0-SNAPSHOT.jar server config.yml
...I get NumberFormatException:
java.lang.NumberFormatException: For input string: ">"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Integer.parseInt(Integer.java:580)
at java.lang.Integer.parseInt(Integer.java:615)
at ch.qos.logback.core.pattern.FormatInfo.valueOf(FormatInfo.java:73)
at ch.qos.logback.core.pattern.parser.Parser.T(Parser.java:136)
at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:96)
at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
at ch.qos.logback.core.pattern.parser.Parser.parse(Parser.java:91)
at ch.qos.logback.core.pattern.PatternLayoutBase.start(PatternLayoutBase.java:83)
at io.dropwizard.logging.AbstractAppenderFactory.buildLayout(AbstractAppenderFactory.java:243)
at io.dropwizard.logging.AbstractOutputStreamAppenderFactory.build(AbstractOutputStreamAppenderFactory.java:25)
at io.dropwizard.logging.DefaultLoggingFactory.configure(DefaultLoggingFactory.java:143)
at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:83)
at io.dropwizard.cli.Cli.run(Cli.java:78)
at io.dropwizard.Application.run(Application.java:93)
at com.nikolas.master_thesis.DropwizardMasterThesisApplication.main(DropwizardMasterThesisApplication.java:35)
I've also looked into this answer, but no luck! Also, I'm running Fedora 31 OS and I'm using Java 8.
Any advice/help is greatly appreciated.
UPDATE1:
I've digged bit deeper into Configuration Docs and found pattern under this link and edited value under logFormat key to be like this:
logFormat: "%h %l %u [%t] '%r' %s %b '%i{Referer}' '%i{User-Agent}'"
Also found out that Logback project usess PatternLayout class and got this output (as starting lines and other lines are as usual) in terminal:
09:27:33,944 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
09:27:33,944 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.groovy]
09:27:33,944 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.xml]
09:27:33,966 |-INFO in ch.qos.logback.classic.BasicConfigurator@316a598d - Setting up default configuration.
09:27:36,978 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@216914 - Propagating INFO level on Logger[ROOT] onto the JUL framework
09:27:37,008 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@216914 - Propagating DEBUG level on Logger[com.nikolas.master_thesis] onto the JUL framework
09:27:37,010 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[file-appender] - Setting bufferSize to [8 KB]
09:27:37,027 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1805845895 - No compression will be used
09:27:37,028 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1805845895 - Will use the pattern /home/nikola/Documents/DWApp-LOGs/DW-Bookshop_LOG-%d.log for the active file
09:27:37,050 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - The date pattern is 'yyyy-MM-dd' from file name pattern '/home/nikola/Documents/DWApp-LOGs/DW-Bookshop_LOG-%d.log'.
09:27:37,050 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - Roll-over at midnight.
09:27:37,068 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - Setting initial period to Thu Aug 13 09:27:06 CEST 2020
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - There is no conversion class registered for conversion word [h]
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - [h] is not a valid conversion word
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - There is no conversion class registered for conversion word [l]
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - [l] is not a valid conversion word
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - There is no conversion class registered for conversion word [u]
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - [u] is not a valid conversion word
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - There is no conversion class registered for conversion word [s]
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - [s] is not a valid conversion word
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - There is no conversion class registered for conversion word [b]
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - [b] is not a valid conversion word
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - There is no conversion class registered for conversion word [i]
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - [i] is not a valid conversion word
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - There is no conversion class registered for conversion word [i]
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - [i] is not a valid conversion word
09:27:37,074 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[file-appender] - Active log file name: /home/nikola/Documents/DWApp-LOGs/DW-Bookshop_LOG.log
09:27:37,074 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[file-appender] - File property is set to [/home/nikola/Documents/DWApp-LOGs/DW-Bookshop_LOG.log]
09:27:37,075 |-INFO in ch.qos.logback.classic.AsyncAppender[async-file-appender] - Attaching appender named [file-appender] to AsyncAppender.
09:27:37,076 |-INFO in ch.qos.logback.classic.AsyncAppender[async-file-appender] - Setting discardingThreshold to 0
09:27:37,077 |-INFO in ch.qos.logback.classic.AsyncAppender[async-console-appender] - Attaching appender named [console-appender] to AsyncAppender.
09:27:37,077 |-INFO in ch.qos.logback.classic.AsyncAppender[async-console-appender] - Setting discardingThreshold to 0
Apperantly I need to create logback.xml file as this line states:
09:27:33,944 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.xml]
Any idea where does that logback.xml file needs to located in MAven project and which value for logFormat should I use then?
Upvotes: 2
Views: 3440
Answers (1)
Abel Matos
Reputation: 96
To log http requests in a file you have to set server: requestLog: config. logging config is used for configurate logs from your application.
Look the documentation for that config and logback-access and logback-access-pattern references at: https://www.dropwizard.io/en/latest/manual/configuration.html#request-log
This is my example:
...
logging:
level: INFO
appenders:
- type: file
currentLogFilename: /logs/api.log
archivedLogFilenamePattern: /logs/api-%d{yyyy-MM-dd}.log.gz
archivedFileCount: 7
queueSize: 2048
loggers:
com.mstn: DEBUG
server:
applicationConnectors:
- type: http
port: 8003
adminConnectors:
- type: http
port: 9003
requestLog:
appenders:
- type: file
currentLogFilename: /logs/api-requests.log
archivedLogFilenamePattern: /logs/api-requests-%d{yyyy-MM-dd}.log.gz
archivedFileCount: 7
queueSize: 2048
database:
...
Upvotes: 5
Related Questions
- How to print server responses using LoggingFeature in Dropwizard 1.0.2?
- Dropwizard RequestLog
- Log file not generated in dropwizard
- Logging in a dropwizard application
- How to log JSON responses in Dropwizard (Jersey)
- Logging configuration for dropwizard to log everything into one file with custom log format
- Dropwizard custom logger does not dump logs into the file
- Logging some request headers with DropWizard
- Implementing custom logging for Dropwizard resources
- Dropwizard Custom log file via Logback