CODEWITHSUNDEEP
javaspring-bootwebsocketjetty-9
aviad cohen
aviad cohen

Reputation: 667

Potential memory leak of websocket sessions in jetty (9.4.15)

I have a service that accept and create several websocket sessions back and forth (they are closed and then reopened etc). For closing the websocket sessions, I am calling the close method the webSocketSession object of spring, which is tied inside, I guess, to jetty's own WebSocketSession object.

My service runs with spring boot (1.5.20) with an embedded jetty server (9.4.15), and is using G1 GC with default parameters (perhaps will be fine tuned later on)

lately, it seems that this service has some memory leak which I am trying to investigate. I have taken a heap dump after verifying from the application POV that I have no open sessions from my side, and after manually "forcing" GC (using JConsole).

The heap dump was taken using JConsole with live=true parameter, so I am most certainly sure that I have a dump after GC had performed all available cleanup on it.

I have analyzed the dump using Eclipse MAT, and found out that I have ~8K WebSocketSession objects that are kept alive. This metric is somehow proportional to an applicative metric I have showing that my service dealt with so far with ~8K connections (8K of incoming WS connections and 8K of outgoing WS connections where I am using jetty as a WS client).

I have analyzed the path to GC roots to one of these objects, and found that it is being held, among others, by some of Jetty threads (qtp-XXX threads) and by some schedulers (see set of images below) enter image description here enter image description here enter image description here enter image description here My questions:

  1. Does any one came across such behavior of jetty with problem releasing websockets resources over time ?
  2. I see some scheduler threads that also holds the websocket sessions. Does any one knows if these are internals of jetty ? or are they contributed from spring boot layer ? or something else ?
  3. Could it be that Jetty holds all sessions internally (even the ones that were closed) and only periodically releasing them ? (or some sort of mechanism that would explain how I have these 8K supposedly closed sessions live and active ?)

Upvotes: 2

Views: 1597

Answers (1)

user12661762
user12661762

Reputation: 11

Yes, I've experienced similar problem. After upgrading to jetty 9.4.28 the memory leak has gone.

Upvotes: 1

Related Questions