Get the full, raw HTTP request message that would be sent from an HttpPost

Question

I'm trying to build a raw HTTP POST request. I don't want to actually connect to the server and send the message, however.

I've been poking around the Apache HTTP libraries, hoping that I could just create an HttpPost object, set the entity, and then grab the message that it would have created. So far, I can dump the entity, but not the entire request as it'd appear on the server-side.

Any ideas? Aside from just re-creating the wheel, of course.

Solution

I refactored ShyJ's response into a pair of static classes, however the original response works just fine. Here are the two classes:

public static final class LoopbackPostMethod extends PostMethod {
    private static final String STATUS_LINE = "HTTP/1.1 200 OK";

    @Override
    protected void readResponse(HttpState state, HttpConnection conn) throws IOException, HttpException {
        statusLine = new StatusLine (STATUS_LINE);
    }
}

public static final class LoopbackHttpConnection extends HttpConnection {
    private static final String HOST = "127.0.0.1";
    private static final int PORT = 80;

    private final OutputStream fOutputStream;

    public LoopbackHttpConnection(OutputStream outputStream) {
        super(HOST, PORT);
        fOutputStream = outputStream;
    }

    @Override
    public void flushRequestOutputStream() throws IOException { /* do nothing */ }

    @Override
    public OutputStream getRequestOutputStream() throws IOException, IllegalStateException {
        return fOutputStream;
    }

    @Override
    public void write(byte[] data) throws IOException, IllegalStateException {
        fOutputStream.write(data);
    }
}

Here's the factory method that I'm using for my own implementation, as an example:

private ByteBuffer createHttpRequest(ByteBuffer data) throws HttpException, IOException {
    LoopbackPostMethod postMethod = new LoopbackPostMethod();
    final ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
    postMethod.setRequestEntity(new ByteArrayRequestEntity(data.array()));
    postMethod.execute(new HttpState(), new LoopbackHttpConnection(outputStream));
    byte[] bytes = outputStream.toByteArray();
    ByteBuffer buffer = ByteBuffer.allocate(bytes.length);
    buffer.put(bytes);
    return buffer;
}

Answer 1

With HttpClient 4.x

    private static String toRawHttp(HttpUriRequest request) throws IOException {
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        HttpTransportMetricsImpl outTransportMetrics = new HttpTransportMetricsImpl();
        SessionOutputBufferImpl buffer = new SessionOutputBufferImpl(outTransportMetrics, 1);
        HttpMessageWriter<HttpRequest> defaultHttpRequestWriter = DefaultHttpRequestWriterFactory.INSTANCE.create(buffer);
        DefaultHttpRequestWriterFactory writerFactory = new DefaultHttpRequestWriterFactory(null) {
            @Override
            public HttpMessageWriter<HttpRequest> create(SessionOutputBuffer buffer) {
                return defaultHttpRequestWriter;
            }
        };
        buffer.bind(baos);

        ConnectionConfig config = ConnectionConfig.DEFAULT;
        DefaultBHttpClientConnection connection = new DefaultBHttpClientConnection(config.getBufferSize(),
                                                                                   config.getFragmentSizeHint(),
                                                                                   ConnSupport.createDecoder(config),
                                                                                   ConnSupport.createEncoder(config),
                                                                                   config.getMessageConstraints(),
                                                                                   null,
                                                                                   null,
                                                                                   writerFactory,
                                                                                   null) {
            @Override
            protected void ensureOpen() {
                //using writerFactory buffer instead of socket
            }

            @Override
            protected OutputStream createOutputStream(long len, SessionOutputBuffer outbuffer) {
                if (len == ContentLengthStrategy.CHUNKED) {
                    return new ChunkedOutputStream(2048, buffer);
                } else if (len == ContentLengthStrategy.IDENTITY) {
                    return new IdentityOutputStream(buffer);
                } else {
                    return new ContentLengthOutputStream(buffer, len);
                }
            }
        };

        CloseableHttpClient client = HttpClients.custom()
                                                .setRequestExecutor(new HttpRequestExecutor() {

                                                    @Override
                                                    protected HttpResponse doSendRequest(HttpRequest request, HttpClientConnection conn, HttpContext context) throws IOException, HttpException {
                                                        // inject fake connection
                                                        return super.doSendRequest(request, connection, context);
                                                    }

                                                    @Override
                                                    protected HttpResponse doReceiveResponse(HttpRequest request, HttpClientConnection conn, HttpContext context) {
                                                        return new BasicHttpResponse(request.getProtocolVersion(), 200, "OK");
                                                    }
                                                })
                                                .build();
        client.execute(request);
        return new String(baos.toByteArray());
    }

Using:

 RequestBuilder builder = RequestBuilder.post(requestModel.getUrl()).addHeader("X-Hello","Word");
 System.out.println(toRawHttp(builder.build()));

Will print:

POST /ff HTTP/1.1
X-Hello: Word
Host: localhost:8080
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.5.8 (Java/1.8.0_161)
Accept-Encoding: gzip,deflate

Answer 2

This can be achived with http-client and faking some methods. I used the 3.1 version of http-client.

Example

This code:

import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.OutputStream;

import org.apache.commons.httpclient.HttpConnection;
import org.apache.commons.httpclient.HttpException;
import org.apache.commons.httpclient.HttpState;
import org.apache.commons.httpclient.StatusLine;
import org.apache.commons.httpclient.methods.PostMethod;

public class Main {
    public static void main(String[] args) throws Exception {
        final ByteArrayOutputStream baos = new ByteArrayOutputStream();
        PostMethod method = new PostMethod () {
            @Override
            protected void readResponse(HttpState state, HttpConnection conn)
                    throws IOException, HttpException {
                statusLine = new StatusLine ("HTTP/1.1 200 OK");
            }
        };
        method.addParameter("aa", "b");

        method.execute(new HttpState (), new HttpConnection("http://www.google.abc/hi", 80) {

            @Override
            public void flushRequestOutputStream() throws IOException {
            }

            @Override
            public OutputStream getRequestOutputStream() throws IOException,
                    IllegalStateException {
                return baos;
            }

            @Override
            public void write(byte[] data) throws IOException,
                    IllegalStateException {
                baos.write(data);
            }

        });

        final String postBody = new String (baos.toByteArray());

        System.out.println(postBody);
    }
}

will return

POST / HTTP/1.1
User-Agent: Jakarta Commons-HttpClient/3.1
Host: http://www.google.abc/hi
Content-Length: 4
Content-Type: application/x-www-form-urlencoded

aa=b

Answer 3

What I would do is implement one or more of HttpClient's interfaces, and use my no-op implementations.

Look at ClientConnectionManager and AbstractHttpClient, for example.any connection.