Capturing HTML generated from ASP.NET

Question

How do I best capture the HTML (in my instance, for logging) rendered by an aspx-page?

I dont want to have to write back to the page using Response.Write, since it messes up my site layout.

Using the Response.OutputStream or Response.Output's stream results in an ArgumentException ({System.ArgumentException: Stream was not readable.)

Answer 1

Good question, i had to try out and see if i could create a HttpModule to do what you are describing.

I didnt have any luck trying to read from the responsestream, but using the ResponseFilter gave me a way to capture the content.

The following code seems to work pretty good, and i figured maybe you could use the code as a base. But remember this is just something i threw together fast, it has not been tested in any way. So dont use it in any production environment without proper reviewing/testing and such. Feel free to comment on it though ;)

public class ResponseLoggerModule : IHttpModule
{
    private class ResponseCaptureStream : Stream
    {
        private readonly Stream _streamToCapture;
        private readonly Encoding _responseEncoding;

        private string _streamContent;
        public string StreamContent
        {
            get { return _streamContent; }
            private set
            {
                _streamContent = value;
            }
        }

        public ResponseCaptureStream(Stream streamToCapture, Encoding responseEncoding)
        {
            _responseEncoding = responseEncoding;
            _streamToCapture = streamToCapture;

        }

        public override bool CanRead
        {
            get { return _streamToCapture.CanRead; }
        }

        public override bool CanSeek
        {
            get { return _streamToCapture.CanSeek; }
        }

        public override bool CanWrite
        {
            get { return _streamToCapture.CanWrite; }
        }

        public override void Flush()
        {
            _streamToCapture.Flush();
        }

        public override long Length
        {
            get { return _streamToCapture.Length; }
        }

        public override long Position
        {
            get
            {
                return _streamToCapture.Position;
            }
            set
            {
                _streamToCapture.Position = value;
            }
        }

        public override int Read(byte[] buffer, int offset, int count)
        {
            return _streamToCapture.Read(buffer, offset, count);
        }

        public override long Seek(long offset, SeekOrigin origin)
        {
            return _streamToCapture.Seek(offset, origin);
        }

        public override void SetLength(long value)
        {
            _streamToCapture.SetLength(value);
        }

        public override void Write(byte[] buffer, int offset, int count)
        {
            _streamContent += _responseEncoding.GetString(buffer);
            _streamToCapture.Write(buffer, offset, count);
        }

        public override void Close()
        {
            _streamToCapture.Close();
            base.Close();
        }
    }

    #region IHttpModule Members

    private HttpApplication _context;
    public void Dispose()
    {

    }

    public void Init(HttpApplication context)
    {
        _context = context;

        context.PreRequestHandlerExecute += new EventHandler(context_PreRequestHandlerExecute);
        context.PreSendRequestContent += new EventHandler(context_PreSendRequestContent);
    }

    void context_PreRequestHandlerExecute(object sender, EventArgs e)
    {
        _context.Response.Filter = new ResponseCaptureStream(_context.Response.Filter, _context.Response.ContentEncoding);
    }

    void context_PreSendRequestContent(object sender, EventArgs e)
    {
        ResponseCaptureStream filter = _context.Response.Filter as ResponseCaptureStream;

        if (filter != null)
        {
            string responseText = filter.StreamContent;

            // Logging logic here
        }
    }

    #endregion
}

Answer 2

Many load testers will allow you to log the HTTP responses generated, but bear in mind with ASP.NET those could be some very large log-files.

Edit: Response.Filter as per Tom Jelen's code is designed to give this kind of oversight and Response.Outputstream is otherwise unreadable.

Edit 2: For a page rather than a HTTPModule

public class ObserverStream : Stream
{
  private byte[] buffer = null;
  private Stream observed = null;

  public ObserverStream (Stream s)
  {
    this.observed = s;
  }

  /* important method to extend #1 : capturing the data */
  public override void Write(byte[] buffer, int offset, int count)
  {
    this.observed.Write(buffer, offset, count);
    this.buffer = buffer; //captured!
  }

  /* important method to extend #2 : doing something with the data */
  public override void Close()
  {
    //this.buffer available for logging here!
    this.observed.Close();
  }

  /* override all the other Stream methods/props with this.observed.method() */

  //...

}

and in your Page_Load (or before your response is written anyway)

Response.Filter = new ObserverStream(Response.Filter);

Answer 3

One way to to make server-side XMLHTTP request to your own server. Grab the result and save it to a file or DB.

Alternately you can use AJAX on the client, grab the result, and POST it back to the server.