Reading Ring request body when already read

Question

My question is, how can I idiomatically read the body of a Ring request if it has already been read?

Here's the background. I'm writing an error handler for a Ring app. When an error occurs, I want to log the error, including all relevant information that I might need to reproduce and fix the error. One important piece of information is the body of the request. However, the statefulness of the :body value (because it is a type of java.io.InputStream object) causes problems.

Specifically, what happens is that some middleware (the ring.middleware.json/wrap-json-body middleware in my case) does a slurp on the body InputStream object, which changes the internal state of the object such that future calls to slurp return an empty string. Thus, the [content of the] body is effectively lost from the request map.

The only solution I can think of is to preemptively copy the body InputStream object before the body can be read, just in case I might need it later. I don't like this approach because it seems clumsy to do some work on every request just in case there might be an error later. Is there a better approach?

Answer 1

I adopted @noisesmith's basic approach with a few modifications, as shown below. Each of these functions can be used as Ring middleware.

(defn with-request-copy
  "Transparently store a copy of the request in the given atom.
  Blocks until the entire body is read from the request.  The request
  stored in the atom (which is also the request passed to the handler)
  will have a body that is a fresh (and resettable) ByteArrayInputStream
  object."
  [handler atom]
  (fn [{orig-body :body :as request}]
    (let [{body :stream} (groundhog/tee-stream orig-body)
          request-copy (assoc request :body body)]
      (reset! atom request-copy)
      (handler request-copy))))

(defn wrap-error-page
  "In the event of an exception, do something with the exception
  (e.g. report it using an exception handling service) before
  returning a blank 500 response.  The `handle-exception` function
  takes two arguments: the exception and the request (which has a
  ready-to-slurp body)."
  [handler handle-exception]
  ;; Note that, as a result of this top-level approach to
  ;; error-handling, the request map sent to Rollbar will lack any
  ;; information added to it by one of the middleware layers.
  (let [request-copy (atom nil)
        handler (with-request-copy handler request-copy)]
    (fn [request]
      (try
        (handler request)
        (catch Throwable e
          (.reset (:body @request-copy))
          ;; You may also want to wrap this line in a try/catch block.
          (handle-exception e @request-copy)
          {:status 500})))))

Answer 2

I think you're stuck with some sort of "keep a copy around just in case" strategy. Unfortunately it looks like :body on the request must be an InputStream and nothing else (on the response it can be a String or other things which is why I mention it)

Sketch: In a very early middleware, wrap the :body InputStream in an InputStream that resets itself on close (example). Not all InputStreams can be reset, so you may need to do some copying here. Once wrapped, the stream can be re-read on close, and you're good. There's memory risk here if you have giant requests.

Update: here's an half-baked attempt, inspired in part by tee-stream in groundhog.

(require '[clojure.java.io :refer [copy]])
(defn wrap-resettable-body
  [handler]
  (fn [request]
    (let [orig-body (:body request)
          baos (java.io.ByteArrayOutputStream.)
          _ (copy orig-body baos)
          ba (.toByteArray baos)
          bais (java.io.ByteArrayInputStream. ba)
          ;; bais doesn't need to be closed, and supports resetting, so wrap it
          ;; in a delegating proxy that calls its reset when closed.
          resettable (proxy [java.io.InputStream] []
                       (available [] (.available bais))
                       (close [] (.reset bais))
                       (mark [read-limit] (.mark bais read-limit))
                       (markSupported [] (.markSupported bais))
                       ;; exercise to reader: proxy with overloaded methods...
                       ;; (read [] (.read bais))
                       (read [b off len] (.read bais b off len))
                       (reset [] (.reset bais))
                       (skip [n] (.skip bais)))
          updated-req (assoc request :body resettable)]
      (handler updated-req))))

Answer 3

I have a lib that sucks up the body, replaces it with a stream with identical contents, and stores the original so that it can be deflated later.

groundhog

This is not adequate for indefinitely open streams, and is a bad idea if the body is the upload of some large object. But it helps for testing, and recreating error conditions as a part of the debugging process.

If all you need is a duplicate of the stream, you can use the tee-stream function from groundhog as the basis for your own middleware.