STOMP with RabbitMQ and Springboot - Multitenancy via Vhosts

Question

We are currently trying to implement multitenancy across our messaging system. Our stack includes a springboot microservice attached to a rabbitMQ broker and redis to store room-specific data in memory for fast retrieval

Usually you statically declare your broker configuration in the class WebSocketConfig implements WebSocketMessageBrokerConfigurer e. g.:

 public void configureMessageBroker(MessageBrokerRegistry registry) {
            registry.enableStompBrokerRelay(SYNC_CHANNEL_NAME)
                    .setSystemLogin(syncProperties.getBroker().getUser())
                    .setSystemPasscode(syncProperties.getBroker().getPassword())
                    .setRelayHost(syncProperties.getBroker().getHost())
                    .setRelayPort(syncProperties.getBroker().getPort())
                    .setUserDestinationBroadcast(SYNC_CHANNEL_NAME + "unresolved-user")
                    .setUserRegistryBroadcast(SYNC_CHANNEL_NAME + "sync-user-reg")
                    .setTaskScheduler(messageBrokerTaskScheduler)
                    .setVirtualHost("tenant1")  // HERE setting virtual host statically
                    .setSystemHeartbeatSendInterval(syncProperties.getBroker().getSystemHeartbeatSendInterval())
                    .setSystemHeartbeatReceiveInterval(syncProperties.getBroker().getSystemHeartbeatReceiveInterval());
            [...]
 

The issue with such approach is that this is one static configuration for the entire service.

In that setup our broker works as intended though, routing all the messages through the correct virtual host but we need to create a configuration that accepts multiple virtual hosts.

Meaning we can not statically set a virtual host in the configuration and decide on which virtual host to establish the connection based on the users credentials.

What I found so far suggests that the virtual host should be set either on the client side or alternatively should be set on the CONNECT frame.

So this is what we use at the moment, we intercept connection attempts from clients, parse and validate their JWT, extract information from which virtual host said user is and modify the host header in the intercepted CONNECT frame before the connection gets established and the connection upgraded to websocket.

See TenantRoutingInterceptor which we register to the Inbound Channel:

public class TenantRoutingInterceptor implements ChannelInterceptor {
    @Override
    public Message<?> preSend(Message<?> message, MessageChannel channel) {
        StompHeaderAccessor accessor = MessageHeaderAccessor.getAccessor(message, StompHeaderAccessor.class);
        if (accessor == null) {
            return message;
        }
 
        if (StompCommand.CONNECT.equals(accessor.getCommand())) {
            if (sessionAttributes.containsKey("TENANT_ID")) {
                String tenantId = (String) sessionAttributes.get("TENANT_ID");
                accessor.setHost(tenantId);
        }
 
        return message;
    }
}

What I've read about this is: With the virtual host being set/specified in the CONNECT/STOMP frame's headers for connection establishment should be sufficient. After the connection is established, all subsequent frames will operate within that virtual host context.

But it seems that some messages get lost in the process which is why I am trying to find help.

While we use this approach (no fixed virtual host set on configureMessageBroker and handling that through the interceptor) a certain error pattern is emerging:

The connection gets established successfully on the right virtual host (as can be observed in the RabbitMQ admin dashboard under "Connections") but some messages are lost resulting in faulty web client behaviour.

We have also logged the normal behaviour on root vhost (everything working as intended) and the tenant vhost which show the following abnormalities:

• Initial connection and scope creation works normally
• Scope subscriptions are sent successfully
• BUT: No scope updates are being received
• Messages to /app/scope.apply are missing in the tenant log

Other noteworthy information: We made sure to create those virtual hosts in advance on the RabbitMQ broker and granted our test user (guest/guest) all the rights on all virtual host (configure,read,write) so permission issues can be outruled.

Any idea why this is happening? As mentioned if I test this in isolation for a single tenant and set it fixed in configureMessageBroker() everything works as intended, but this locks me into having just this one tenant available for that running service.