How to integrate stomp/websockets using spring boot messaging with Jetty 10?

Question

Some backstory:

I am working on a spring boot application which uses the spring-boot-starter-websocket library to update a frontend client with messages using STOMP.

The setup is as per spring docs - using WebSocketMessageBrokerConfigurer.

@Configuration
@EnableWebSocketMessageBroker
@RequiredArgsConstructor
public class WebSocketConfig implements WebSocketMessageBrokerConfigurer {

    private static final long[] HEARTBEAT = {
            WEBSOCKETS_OUTCOMING_BROKER_HEARTBEAT_MIN_INTERVAL,
            WEBSOCKETS_INCOMING_CLIENT_HEARTBEAT_MIN_INTERVAL
    };

    private final CorsConfig corsConfig;
    private final TaskScheduler scheduler;

    @Override
    public void configureMessageBroker(MessageBrokerRegistry config) {
        config.enableSimpleBroker(WebSocketPath.TOPIC_PREFIX)
                .setHeartbeatValue(HEARTBEAT)
                .setTaskScheduler(scheduler);
        config.setApplicationDestinationPrefixes(WebSocketPath.APP_PREFIX, WebSocketPath.USER_PREFIX);
    }

    @Override
    public void registerStompEndpoints(StompEndpointRegistry registry) {
        registry.addEndpoint(WebSocketPath.WS_ENDPOINT)
                .setAllowedOrigins(corsConfig.getAllowedOrigins().toArray(new String[0]));
    }

    @Bean
    public ServletServerContainerFactoryBean servletServerContainerFactoryBean() {
        ServletServerContainerFactoryBean servletServerContainerFactoryBean = new ServletServerContainerFactoryBean();
        servletServerContainerFactoryBean.setMaxTextMessageBufferSize(WEBSOCKETS_BUFFER_MAX_SIZE); //chars
        servletServerContainerFactoryBean.setMaxBinaryMessageBufferSize(WEBSOCKETS_BUFFER_MAX_SIZE); //bytes
        servletServerContainerFactoryBean.setMaxSessionIdleTimeout(WEBSOCKETS_SESSION_TIMEOUT);
        return servletServerContainerFactoryBean;
    }

}

I can see from Spring docs that this ServletServerContainerFactoryBean setup does config for embedded Tomcat and NOT Jetty.

This app is run locally for testing on embedded Tomcat, and stomp heartbeats every 120 seconds between it and the frontend client work as they should; websocket connection stays open.

In staging/prod, the app is put onto a Jetty server as a .war file. Up until recently, it was running Jetty9.4 and there were no websocket issues.

Now that server has been updated to run Jetty10, and I gather Jetty changed its default idleTimeout between 9.4 and 10; down to 30 secs from 5 mins. This means the heartbeats aren't frequent enough any more, and the websocket connection is cut every 30 secs. Running in debug I can see this:

04 Jul 2023 09:53:30.417 [Scheduler-1882554559-1] DEBUG o.s.w.s.h.LoggingWebSocketHandlerDecorator - Transport error in JettyWebSocketSession[id=08b189c1-8559-b740-229f-8b8e4d92a0f4, uri=ws://my-example.com/api/v1/ws-endpoint]
org.eclipse.jetty.websocket.api.exceptions.WebSocketTimeoutException: Connection Idle Timeout
    at org.eclipse.jetty.websocket.common.JettyWebSocketFrameHandler.convertCause(JettyWebSocketFrameHandler.java:524)
    at org.eclipse.jetty.websocket.common.JettyWebSocketFrameHandler.onError(JettyWebSocketFrameHandler.java:258)
    at org.eclipse.jetty.websocket.core.internal.WebSocketCoreSession.lambda$closeConnection$2(WebSocketCoreSession.java:284)
    at org.eclipse.jetty.server.handler.ContextHandler.handle(ContextHandler.java:1468)
    at org.eclipse.jetty.server.handler.ContextHandler.handle(ContextHandler.java:1487)
    at org.eclipse.jetty.websocket.core.server.internal.AbstractHandshaker$1.handle(AbstractHandshaker.java:212)
    at org.eclipse.jetty.websocket.core.internal.WebSocketCoreSession.closeConnection(WebSocketCoreSession.java:284)
...

Updating properties in Jetty's .ini files (jetty.ssl.idleTimeout and jetty.http.idleTimeout) has no effect on changing the idle timeout.

I assume this is due to the classes being used inside the spring websocket library being Jetty9.4 based rather than Jetty10, somehow.

springBootVersion = 2.7.5 javaVersion = 17

I tried two approaches to fix it:

1. Try a quick n dirty way to override the current setup to have a 5 min idleTimeout again

Changing things within the app code:

I tried using Spring's instructions for excluding Jetty 9 but that left me no access to the policy configuration method which is the documented way to change the Tomcat servletServerContainerFactoryBean config for Jetty, as linked above. With Jetty 9.4 you can *supply a pre-configured Jetty WebSocketServerFactory and plug that into Spring’s DefaultHandshakeHandler through your WebSocket Java config * but this isn't the same in Jetty 10 and I cannot for the life of me find a workaround that fits in.

I did then read that if I'm not going to be using embedding Jetty, spring docs say that websocket config is assumed to be done by the container. Spring Boot provides WebSockets auto-configuration for embedded Tomcat, Jetty, and Undertow. If you deploy a war file to a standalone container, Spring Boot assumes that the container is responsible for the configuration of its WebSocket support.

So, I thought maybe I'll get lucky just changing settings on jetty container: Tried updating properties in the .ini files and restarting; no change. To check I wasn't going mad I overrode an http port property too and that worked; so the ini changes are being picked up, just that neither of jetty.ssl.idleTimeout and jetty.http.idleTimeout could do the trick.

2. Try to replace the WebSocketConfig class above with Javax or Jetty's own Jetty10 compatible websocket libraries

This seems the better approach long term, but I am new to websockets & stomp, as well as new to this app, and I do not know how to adapt the Jetty 10 websocket examples I can find to work with the Spring Message Broker logic that's already there; or what to replace that with if it'd have to be ripped out too. I can't find any documentation around it. Plus, I am operating on the assumption that using the embedded examples will also work for a deployed war file.

For now, we've had to decrease the heartbeats to be <30 seconds to avoid dropping connections, but really would like to go back up to the 2 minute region, so I do want to get this sorted.

Short of containerising the app with embedded Tomcat (which would cause issues in other places), what's the best way to sort this out?