How to Override request logging mechanism in Jetty 11.0.6

Question

I am updating jetty 9 to jetty11 and I see that, org.eclipse.jetty.server.AsyncNCSARequestLog is deprecated and The other two classes are introduced for this - org.eclipse.jetty.server.CustomRequestLog and org.eclipse.jetty.server.AsyncRequestLogWriter.

But the problem is I used to extend AsyncNCSARequestLog class and override log and write methods(see below code for more info) but couldn't find the way to do it in Jetty11.

public class JettyRequestLogger extends AsyncNCSARequestLog
    {
        private Request request;
    
        @Override
        public void log(Request request, Response response)
        {
            this.request = request;
            super.log(request, response);
        }
    
        @Override
        public void write(String logStr) throws IOException
        {
            logStr = setCustomAttributesToLog(logStr);
            super.write(logStr);
        }
       private String setCustomAttributesToLog(String logStr)
       {
         // logic here
       }

    }

Update : My use case is to add custom attributes like UserId, UserName, and custom attributes etc while printing the jetty logs

Answer 1

There's been some important changes in RequestLog as a whole since Jetty 9.x

RequestLog implementations are added to the Server now, don't use the old school RequestLogHandler.

// Modern version (for all system loads and speeds)
Slf4jRequestLogWriter slfjRequestLogWriter = new Slf4jRequestLogWriter();
slfjRequestLogWriter.setLoggerName("com.company.request.log");
String format = "%{client}a - %u %t '%r' %s %O '%{Referer}i' '%{User-Agent}i' '%C'";
CustomRequestLog customRequestLog = new CustomRequestLog(slfjRequestLogWriter, format);
server.setRequestLog(customRequestLog);

// Old school version (not for busy systems)
Path outputPattern = logsDir.resolve("yyyy_mm_dd.request.log");
AsyncRequestLogWriter logWriter = new AsyncRequestLogWriter(outputPattern.toString());
logWriter.setFilenameDateFormat("yyyy_MM_dd");
logWriter.setRetainDays(90);
logWriter.setTimeZone("GMT");
String format = "%{client}a - %u %t '%r' %s %O '%{Referer}i' '%{User-Agent}i' '%C'";
CustomRequestLog requestLog = new CustomRequestLog(logWriter, format);
server.setRequestLog(requestLog);

While the RequestLogHandler still exists, it has some fundamental flaws in that it does not log requests that fail during parsing step, or requests that fail during the compliance validation steps, or requests that fail during the ssl/tls validation steps, or requests that don't belong to a known context, or exchanges that fail during response generation, or exchanges that pass through the error handling layers back to the application, but fail at the application layer, etc.. As you can see, there were many many edge cases that RequestLogHandler just could not handle, so the fundamental Server.setRequestLog(RequestLog) was created, use it.

Internal changes in Request and Response have been made to hold on to the state of those components when each component was "committed", so you have to use those new methods from Request and Response to get accurate information about the state of those objects when they were actually processed by the network layer. Using the original methods can easily result in bad/inaccurate data due to when the logging occurs during the http exchange and lifecycle.

RequestLog implementations that Jetty ships consist of two halves.

The RequestLog itself a component that accepts the Request and Response and formats the information into a log line String.

The RequestLog.Writer component is used by the RequestLog to writes the raw string to whatever is necessary.

• Slf4jRequestLogWriter (default implementation, and recommended) - writes the string to a slf4j named logger. This is the best choice for setting up unique log files, rolling behavior (be it by size, or date, or arbitrary duration, or other trigger), old log archiving (with compression), alerts, supporing extended access log analysis tools, etc.
• RequestLogWriter (old school, many gotchas) - synchronously write to any arbitrary OutputStream (You can use of Jetty's old school RolloverFileOutputStream here). This will write every entry once it's received, if you have more network events than your disk I/O can handle (a surprisingly common scenario) switch to writing in bursts with the async version.
• AsyncRequestLogWriter - async version of RequestLogWriter, if you still have more network events than your disk I/O can handle, switch to the Slf4jRequestLogWriter and use the various slf4j implementation's advanced async log appending, which is far more reliable than the simple java.io behaviors than Jetty itself uses in the AsyncRequestLogWriter.

The most common form of customized Request Logging is to alter WHAT is logged, vs customizing what the output format looks like (which is already handled by the CustomRequestLog itself).

An example would be to only log error cases, and not log the rest.

Note that this shows an example of the "committed" state information mentioned above.

package jetty;

import org.eclipse.jetty.server.CustomRequestLog;
import org.eclipse.jetty.server.Request;
import org.eclipse.jetty.server.Response;

public class ErrorOnlyRequestLog extends CustomRequestLog
{
    public ErrorOnlyRequestLog(Writer writer, String formatString)
    {
        super(writer, formatString);
    }

    public ErrorOnlyRequestLog(String file)
    {
        super(file);
    }

    public ErrorOnlyRequestLog(String file, String format)
    {
        super(file, format);
    }

    @Override
    public void log(Request request, Response response)
    {
        // Get the response status actually sent (as you cannot rely on
        // the Response.getStatus() at this point in time, which can change
        // due to a number of factors, including error handling, dispatch
        // completion, recycling, etc)
        int committedStatus = response.getCommittedMetaData().getStatus();

        // only interested in error cases - bad request & server errors
        if ((committedStatus >= 500) || (committedStatus == 400))
        {
            super.log(request, response);
        }
        else
        {
            System.err.println("### Ignored request (response.committed.status=" + committedStatus + "): " + request);
        }
    }
}