How to log all the request-responses in spring rest?

Question

The application should log the following information without impacting a client, asynchronously(in a separate thread).

• Request HTTP Method and URI
• Request Headers (Except the default)
• Client's IP Address
• Request Processing Time(In milliseconds)
• Request Body
• Response Body

If we consume inputstream in the filter, then it cant be consumed again by spring for json to object mapping. Somewhere during the input stream to object mapping, can we plug our logger?

Update:

We can write over logging code in a MessageConverter, but it doesnt seems to be a good idea.

public class MyMappingJackson2MessageConverter extends AbstractHttpMessageConverter<Object> {
    ...
    protected Object readInternal(Class<? extends Object> clazz, HttpInputMessage inputMessage)
            throws IOException, HttpMessageNotReadableException {
        InputStream inputStream = inputMessage.getBody();
        String requestBody = IOUtils.toString(inputStream, StandardCharsets.UTF_8);
        String method = request.getMethod();
        String uri = request.getRequestURI();
        LOGGER.debug("{} {}", method, uri);
        LOGGER.debug("{}", requestBody);
        return objectMapper.readValue(requestBody, clazz);
    }

    protected void writeInternal(Object o, HttpOutputMessage outputMessage)
            throws IOException, HttpMessageNotWritableException {
        String responseBody = objectMapper.writeValueAsString(o);
        LOGGER.debug("{}", responseBody);
        outputMessage.getBody().write(responseBody.getBytes(StandardCharsets.UTF_8));
    }
}

Answer 1

Here is an workable example to log request and response bodies.

@Configuration
@Slf4j
public class CustomRestFilterBean extends GenericFilterBean {

    @Override
    public void doFilter(ServletRequest request, ServletResponse response, FilterChain filterChain)
            throws IOException, ServletException {
        // lid is a log id. It will help us in logs to find all records from one request.
        String lid = ObjectUtils.getIdentityHexString(new Object());
        request.setAttribute(Header.LID, lid);
        HttpServletRequestWrapperWithBodyLogger requestWrapper = new HttpServletRequestWrapperWithBodyLogger(
                lid, (HttpServletRequest) request);
        HttpServletResponseWrapperWithBodyLogger responseWrapper = new HttpServletResponseWrapperWithBodyLogger(
                lid, (HttpServletResponse) response);
        filterChain.doFilter(requestWrapper, responseWrapper);
    }

    private static final class HttpServletRequestWrapperWithBodyLogger extends HttpServletRequestWrapper {
        private final String lid;

        public HttpServletRequestWrapperWithBodyLogger(String lid, HttpServletRequest request) {
            super(request);
            this.lid = lid;
        }

        @Override
        public ServletInputStream getInputStream() throws IOException {
            ServletInputStream servletInputStream = super.getInputStream();
            byte[] body = servletInputStream.readAllBytes();
            log.info("[{}] Request body={}.", lid, new String(body, StandardCharsets.UTF_8));
            final ByteArrayInputStream bais = new ByteArrayInputStream(body);
            return new ServletInputStream() {
                @Override
                public boolean isFinished() {
                    return servletInputStream.isFinished();
                }

                @Override
                public boolean isReady() {
                    return servletInputStream.isReady();
                }

                @Override
                public void setReadListener(ReadListener readListener) {
                    servletInputStream.setReadListener(readListener);
                }

                @Override
                public int read() {
                    return bais.read();
                }
            };
        }
    }

    private static final class HttpServletResponseWrapperWithBodyLogger extends HttpServletResponseWrapper {
        private final String lid;
        private final ServletOutputStream servletOutputStream;

        public HttpServletResponseWrapperWithBodyLogger(String lid, HttpServletResponse response) throws IOException {
            super(response);
            this.lid = lid;
            this.servletOutputStream = response.getOutputStream();
        }

        @Override
        public ServletOutputStream getOutputStream() throws IOException {
            return new ServletOutputStream() {
                @Override
                public void write(int b) throws IOException {
                    servletOutputStream.write(b);
                }

                @Override
                public boolean isReady() {
                    return servletOutputStream.isReady();
                }

                @Override
                public void setWriteListener(WriteListener writeListener) {
                    servletOutputStream.setWriteListener(writeListener);
                }

                @Override
                public void write(byte @NotNull [] b, int off, int len) throws IOException {
                    super.write(b, off, len);
                    log.info("[{}] Response body={}.", lid, new String(b, StandardCharsets.UTF_8));
                }
            };
        }
    }
}

Logging output

2024-12-04 15:58:17.075  INFO CustomRestFilterBean: [2df47a8] Request body={
  "fileId": "123",
  "status": "SUCCESS"
}.
2024-12-04 15:58:27.129  INFO CustomRestFilterBean: [2df47a8] Response body={
  "fileName":"misha",
  "size":"256"
 }.

You may also want to add headers. It also can be done via CustomRestFilterBean. But I did it via HandlerInterceptor.

@Configuration
@RequiredArgsConstructor
public class WebMvcConfig {
    @Bean
    public WebMvcConfigurer corsConfigure() {
        return new AppWebMvcConfigurer();
    }

    private static class AppWebMvcConfigurer implements WebMvcConfigurer {

        @Override
        public void addCorsMappings(@NonNull CorsRegistry registry) {
            registry.addMapping("/**")
                    .allowedOrigins("*")
                    .allowedHeaders("*")
                    .allowedMethods("GET", "PUT", "POST", "PATCH", "DELETE", "OPTIONS");
        }

        @Override
        public void addInterceptors(InterceptorRegistry registry) {
            registry.addInterceptor(new RestControllerInterceptor());
        }
    }
}

@Slf4j
@Component
@RequiredArgsConstructor
public class RestControllerInterceptor implements HandlerInterceptor {

    @Override
    public boolean preHandle(@NotNull HttpServletRequest request,
                             @Nullable HttpServletResponse response,
                             @Nullable Object handler) {
        if (isLogging(request)) {
            addLogging(request);
        }
        return true;
    }

    // There is also postHandle method, but I recommend to use afterCompletion as it handles any statuses (2xx, 4xx, 5xx), while postHandle - only success 2xx/ statuses
    @Override
    public void afterCompletion(@NotNull HttpServletRequest request,
                                @NotNull HttpServletResponse response,
                                @NotNull Object handler,
                                @Nullable Exception ex) {
        if (isLogging(request)) {
            String lid = (String) request.getAttribute(Header.LID);
            addLogging(lid, response);
        }
    }

    private boolean isLogging(HttpServletRequest request) {
        return Rest.IGNORED_URLS.stream().noneMatch(url -> new AntPathMatcher().match(url, request.getServletPath()));
    }

    private void addLogging(@NotNull HttpServletRequest request) {
        Enumeration<String> headerNames = request.getHeaderNames();
        log.info("[{}] Request {} {}: headers={}.",
                request.getAttribute(Header.LID),
                request.getMethod(),
                request.getRequestURL(),
                getHeaders(request, headerNames)

        );
    }

    private void addLogging(String lid, HttpServletResponse response) {
        Collection<String> headerNames = response.getHeaderNames();
        log.info("[{}] Response {}: headers={}",
                lid,
                response.getStatus(),
                getHeaders(response, headerNames));
    }

    private List<String> getHeaders(@NotNull HttpServletRequest request, Enumeration<String> headerNames) {
        if (headerNames == null) {
            return null;
        }
        return Collections.list(headerNames).stream()
                .map(request::getHeader)
                .collect(Collectors.toList());
    }

    private List<String> getHeaders(@NotNull HttpServletResponse response, Collection<String> headerNames) {
        if (headerNames == null) {
            return null;
        }
        return headerNames.stream()
                .map(response::getHeader)
                .collect(Collectors.toList());
    }
}

===> Full Logging output

4-12-04 15:58:17.065  INFO RestControllerInterceptor: [2df47a8] Request POST http://localhost:8080/api/v1/file/feedback: headers=[loc
alhost:8080, keep-alive, 77, "Windows", Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36, application/json, "Google C
hrome";v="131", "Chromium";v="131", "Not_A Brand";v="24", application/json, ?0, http://localhost:8080, same-origin, cors, empty, http://localhost:8080/api/v1/swagger-ui/index.html, gzip, deflate, br, zstd, ru-RU,ru;q=0.9,en-US;q=0.8,en;q=0.7,de;q=0.6,bg;q=0.5].
2024-12-04 15:58:17.075  INFO CustomRestFilterBean: [2df47a8] Request body={
  "fileId": "123",
  "status": "SUCCESS"
}.
2024-12-04 15:58:27.129  INFO CustomRestFilterBean: [2df47a8] Response body={
  "fileName":"misha",
  "size":"256"
 }.
2024-12-04 15:58:27.130  INFO RestControllerInterceptor: [2df47a8] Response 200: headers=[Origin, Origin, Origin]

Answer 2

An answer from baeldung.com :

Spring provides a built-in solution to log payloads. We can use ready-made filters by plugging into Spring application using configuration. AbstractRequestLoggingFilter is a filter which provides basic functions of logging. Subclasses should override the beforeRequest() and afterRequest() methods to perform the actual logging around the request. Spring framework provides following concrete implementation classes which can be used to log the incoming request. These are:

• CommonsRequestLoggingFilter
• ServletContextRequestLoggingFilter

Spring Boot application can be configured by adding a bean definition to enable request logging:

@Configuration
public class RequestLoggingFilterConfig {

    @Bean
    public CommonsRequestLoggingFilter logFilter() {
        CommonsRequestLoggingFilter filter
          = new CommonsRequestLoggingFilter();
        filter.setIncludeQueryString(true);
        filter.setIncludePayload(true);
        filter.setMaxPayloadLength(10000);
        filter.setIncludeHeaders(false);
        filter.setAfterMessagePrefix("REQUEST DATA : ");
        return filter;
    }
}

Also, this logging filter requires the log level be set to DEBUG. In application.properties put

logging.level.org.springframework.web.filter.CommonsRequestLoggingFilter=DEBUG

To make the logging asynchronous, we may use asynchronous appenders. Unfortunately it does not support logging response payloads. :(

Answer 3

I would use 2 elements: A LoggingFilter and the Async support from Spring. For the first one, I would use a CommonsRequestLoggingFilter that already knows how to intercept the HTTP requests, create the configuration for that and the Async. You can do something like:

First Enable Async Support

@Configuration
@EnableAsync
public class SpringAsyncConfig { ... }

Then Create the loggingFilter:

public class LoggingFilter extends CommonsRequestLoggingFilter {

@Override
protected void beforeRequest(final HttpServletRequest request, final String message) {
    // DO something
    myAsyncMethodRequest(request, message)
}

@Override
protected void afterRequest(final HttpServletRequest request, final String message) {
    // Do something
   myAsyncMethodResponse(request, message)
}

// -----------------------------------------
// Async Methods
// -----------------------------------------

   @Async
   protected void myAsyncMethodRequest(HttpServletRequest request, String message) {

    // Do your thing
    // You can use message that has a raw message from the properties
   // defined in the logFilter() method. 
   // Also you can extract it from the HttpServletRequest using: 
   // IOUtils.toString(request.getReader());

   }

   @Async
   protected void myAsyncMethodResponse(HttpServletRequest request, String message) {

    // Do your thing
   }

}

Then create a custom logging configuration for the filter that you created:

@Configuration
public class LoggingConfiguration {

    @Bean
    public LoggingConfiguration logFilter() {
        LoggingFilter filter
                = new LoggingFilter();
        filter.setIncludeQueryString(true);
        filter.setIncludePayload(true);
        filter.setIncludeHeaders(true);

        return filter;
    }
}

To extract the data from the request you can use the message parameter or process the HttpServletRequest. Take as an example:

• https://eureka.ykyuen.info/2009/09/27/java-get-the-content-of-inputstream-from-httpservletrequest/
• Get the POST request body from HttpServletRequest

Answer 4

I guess your best option is to do the logging in an Async Method.

@Async
public void asyncMethodWithVoidReturnType() {
  System.out.println("Execute method asynchronously. "
    + Thread.currentThread().getName());
}

Please refer to:

Async

How to Async