CODEWITHSUNDEEP
javajersey
sunil
sunil

Reputation: 601

Jersey response containing incorrect data

Apologies: I don't have a simple test case that reproduces this problem, as it happens very intermittently. However, I would greatly appreciate some help regarding how to even begin diagnosing the issue.

I have a Jersey server running on Tomcat.

When the client makes a request, sometimes a response from a totally different request is mixed in with the correct response.

The "correct" request can be of any kind, but the "bad" response which gets mixed in is always from an SSE stream (EventOutput) or an AsyncResponse.

For example, this is the output received by a client through a normal request:

event: message_sent
id: 1
data: {"value":"hello world"}

{"event-id":"13"}event: message_sent
id: 2
data: {"value":"hello world"}

The genuine response {"event-id":"13"} is present... but surrounding that there are two erroneous SSE events.

The method to handle this request returns simply:

return Response.created(uri).entity(eventId).build();

So I don't understand at which point the unwanted data gets sent (unless Response.created() is returning a response object which had already been used for an SSE stream).

The server logs always show the correct output. We know the client is not at fault, however, as we used a packet sniffer to confirm the responses are malformed.

Notes:

Again, any hints or pointers would be such a great help. I've run out of ideas!

Upvotes: 1

Views: 162

Answers (1)

sunil
sunil

Reputation: 601

After a lot of research, I've concluded that my problem must be (of course) a user error even though I couldn't reproduce the error when I removed the apache proxy from the equation. In my case, I had to be more careful about when considering EventOutputs as closed -- since you can only tell if the connection is open when trying to write to it. In general though, my findings are:

A bug of this kind occurs when you keep references to response objects around, and then write to them after Tomcat has re-used them for another request. This could be a list of e.g. AsyncResponse or EventOutput objects which you need to keep around to resume or write to at a later time.

However, if it's a bug which is difficult to track down and you need a solution, there is a Tomcat setting which will disable the re-use of these objects at the cost of performance, as it says here: https://tomcat.apache.org/tomcat-8.0-doc/security-howto.html

Setting org.apache.catalina.connector.RECYCLE_FACADES system property to true will cause a new facade object to be created for each request. This reduces the chances of a bug in an application exposing data from one request to another.

(Don't be confused by the name; RECYCLE_FACADES=true means that the facades will not get reused, and new ones will get created as they are needed).

There are examples of application bugs like this only manifesting when behind an apache proxy, so if the bug disappears if you access Tomcat directly it doesn't necessarily mean that apache is at fault.

As I say, it's unlikely for this to be caused by a bug in Tomcat, Apache or the mod_proxy_ajp... but if you are an unlucky one, you can try using another connector (mod_jk, for example).

Upvotes: 1

Related Questions