Proxy setting not working with Spring WebClient

Question

My following WebClient is working fine with internet connection but not through our proxy connection.

WebClient webClient = WebClient.builder()
        .baseUrl("https://targetsite.com")
        .build();

webClient.post()
    .uri("/service/serviceName")
    .body(BodyInserters.fromObject(reqData))
    .retrieve()
    .bodyToMono(WebServiceResponse.class)

Event though, the same client is working through proxy, if I set it as mentioned below,

HttpClient httpClient = HttpClient.create()
            .tcpConfiguration(tcpClient -> tcpClient
                .proxy(proxy -> proxy
                    .type(ProxyProvider.Proxy.HTTP)
                    .host("ourproxy.com")
                    .port(8080)));

ReactorClientHttpConnector connector = new ReactorClientHttpConnector(httpClient);

WebClient webClient = WebClient.builder()
        .clientConnector(connector)
        .baseUrl("https://targetsite.com")
        .build();

webClient.post()
    .uri("/service/serviceName")
    .body(BodyInserters.fromObject(reqData))
    .retrieve()
    .bodyToMono(WebServiceResponse.class)

But if I set the same Proxy details either using System.setProperty("http.proxyHost","ourproxy.com"); System.setProperty("http.proxyPort","8080"); or JVM run-time arguments -Dhttp.proxyHost=ourproxy.com -Dhttp.proxyPort=8080

WebClient webClient = WebClient.builder()
        .baseUrl("https://targetsite.com")
        .build();

System.setProperty("http.proxyHost", "ourproxy.com");
System.setProperty("http.proxyPort", "8080");

webClient.post()
    .uri("/service/serviceName")
    .body(BodyInserters.fromObject(reqData))
    .retrieve()
    .bodyToMono(WebServiceResponse.class)

The calls are getting failed with UnknownHostException like,

[04/11/2019 12:32:43.031 IST] DEBUG [reactor-http-epoll-3] [PooledConnectionProvider:254] - Creating new client pool [http] for targetsite.com:443
[04/11/2019 12:32:43.033 IST] DEBUG [reactor-http-epoll-3] [PooledConnectionProvider:254] - [id: 0xe4a0dc15] Created new pooled channel, now 0 active connections and 1 inactive connections
[04/11/2019 12:32:43.045 IST] DEBUG [reactor-http-epoll-3] [SslProvider:254] - [id: 0xe4a0dc15] SSL enabled using engine SSLEngineImpl and SNI targetsite.com:443
[04/11/2019 12:32:43.046 IST] DEBUG [reactor-http-epoll-3] [BootstrapHandlers:254] - [id: 0xe4a0dc15] Initialized pipeline DefaultChannelPipeline{(reactor.left.sslHandler = io.netty.handler.ssl.SslHandler), (reactor.left.sslReader = reactor.netty.tcp.SslProvider$SslReadHandler), (BootstrapHandlers$BootstrapInitializerHandler#0 = reactor.netty.channel.BootstrapHandlers$BootstrapInitializerHandler), (SimpleChannelPool$1#0 = io.netty.channel.pool.SimpleChannelPool$1), (reactor.left.httpCodec = io.netty.handler.codec.http.HttpClientCodec), (reactor.left.decompressor = io.netty.handler.codec.http.HttpContentDecompressor), (reactor.right.reactiveBridge = reactor.netty.channel.ChannelOperationsHandler)}
[04/11/2019 12:32:43.165 IST] ERROR [reactor-http-epoll-2] [AbstractErrorWebExceptionHandler:117] - [13ebf1eb] 500 Server Error for HTTP POST "/service/serviceName"
java.net.UnknownHostException: targetsite.com: Name or service not known

Please help, why my code is not functioning if I set the proxy details via JVM run-time arguments or system properties.

Actually I want to avoid code level proxy setting. So please guide me to correct my code or approach, so that I can use JVM run-time argument option.

Answer 1

If you have an old version of spring-webflux (in my case i had spring-webflux-5.0.6.RELEASE) you can configure the proxy in this way:

ClientHttpConnector connector = new ReactorClientHttpConnector(options -> {
    options.proxy(proxy -> proxy
                          .type(Proxy.HTTP)
                          .address(new InetSocketAddress("Your_IP", 8080)));
  });

WebClient.Builder webClientBuilder =
      WebClient.builder().baseUrl("Your_URL")
          .clientConnector(connector);

Answer 2

Here is what worked for me

Step 1 : Define proxy environment variables

-Dhttp.proxyHost=<proxyHost>
-Dhttp.proxyPort=8080
-Dhttps.proxyHost=<proxyHost>
-Dhttps.proxyPort=8080
-Dhttps.nonProxyHosts=localhost

2. Configuration of proxy on webClient

   @Configuration
   public class WebClientConfiguration {    
    @Bean
    public WebClient webClient() {
        return WebClient.builder() //
                .defaultHeader(ACCEPT, APPLICATION_JSON_VALUE) //
                .clientConnector(new ReactorClientHttpConnector(httpClient())) //
                .build();
    }
   
    private HttpClient httpClient() {
        return HttpClient //
                .create() //
                .proxyWithSystemProperties();
    }
   

   }

3. Set the spring cloud proxy properties (In the application start)

   static {
      String nonProxyHosts = System.getProperty("http.nonProxyHosts");
      if (nonProxyHosts != null) {
        String regexProxyList = nonProxyHosts.replaceAll("\\.", "\\\\.").replaceAll("\\/", "\\\\/").replaceAll("\\*", ".\\*");
        System.setProperty("spring.cloud.gateway.httpclient.proxy.non-proxy-hosts-pattern", regexProxyList);
    }
    String proxyHost = System.getProperty("https.proxyHost");
    String proxyPort = System.getProperty("https.proxyPort");
    if (proxyHost != null && proxyPort != null) {
        System.setProperty("spring.cloud.gateway.httpclient.proxy.host", proxyHost);
        System.setProperty("spring.cloud.gateway.httpclient.proxy.port", proxyPort);
    }
   }
   

Answer 3

I had similar situation and struggled to find what in the end was a stupid mistake :

1. I enabled wiretap :

     val httpClient = HttpClient.create()
                    .wiretap(
                        MyClient::class.java.toString(), LogLevel.DEBUG, AdvancedByteBufFormat.TEXTUAL)
    
                webClient=WebClient.builder()
                    .clientConnector(ReactorClientHttpConnector(httpClient))
                    .codecs(consumerCodecForIncreasedPayloadSize)
                    .build()

2. I was then able to spot something strange in the logs :

    [21119d8d] CONNECT: http://myProxy.my.company.com/<unresolved>:8080

what was that <unresolved> thing when issuing a CONNECT command to the proxy ?

3. then I finally realized : the https.proxyHost value I was setting at startup should not be http://myProxy.my.company.com , but simply myProxy.my.company.com, without the http://

then it worked.

Answer 4

I don't have enough reputation to comment on other answers so i have to start my own answer here.

Since reactor-netty v1.0.8, proxy can be configured from system properties.

Example:

WebClient
    .builder()
    .clientConnector(
        new ReactorClientHttpConnector(
            HttpClient.create().proxyWithSystemProperties()))
.build()

Answer 5

You need to set Proxy Host and Port while creating HttpClient instance per bellow.

HttpClient httpClient =
            HttpClient.create()
                    .proxy(proxy -> proxy.type(ProxyProvider.Proxy.HTTP)
                            .host(sasConfig.getProxyHost())
                            .port(Integer.parseInt(sasConfig.getProxyPort())));

ReactorClientHttpConnector conn = new ReactorClientHttpConnector(httpClient);

WebClient webClient = WebClient.builder().clientConnector(conn).build();

Answer 6

Later when I raised this concern with reactor-netty team(https://github.com/reactor/reactor-netty/issues/887#issuecomment-549439355), it was confirmed by them that, system properties are not supported by reactor.netty.http.client.HttpClient. The only option is setting it through tcpConfiguration, which I mentioned in my question already; so I am closing this question with this remarks.

Thank you all