Reputation: 167
I am using Camel and netty-http to consume the external rest api's of azure.
I have configured camel-cron to run my api 3 hours once, which will trigger azure api to pull all the builds and then for each of the build it will trigger subsequent azure rest api to get further details.
I have implemented this on top of Spring boot and deployed into Tomcat server.
It works well for 2 days then the below error is getting occur while triggering the external rest-api,
Reason: java.lang.IllegalStateException: failed to create a child event loop
at org.apache.camel.support.cache.DefaultProducerCache.acquireProducer(DefaultProducerCache.java:155)
at org.apache.camel.support.cache.DefaultProducerCache.doInAsyncProducer(DefaultProducerCache.java:283)
at org.apache.camel.processor.SendDynamicProcessor.process(SendDynamicProcessor.java:182)
at org.apache.camel.impl.engine.CamelInternalProcessor.process(CamelInternalProcessor.java:388)
at org.apache.camel.processor.Pipeline$PipelineTask.run(Pipeline.java:107)
at org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(DefaultReactiveExecutor.java:181)
at org.apache.camel.impl.engine.DefaultReactiveExecutor.schedule(DefaultReactiveExecutor.java:57)
at org.apache.camel.impl.engine.CamelInternalProcessor$AsyncAfterTask.done(CamelInternalProcessor.java:264)
at org.apache.camel.support.cache.DefaultProducerCache.lambda$doInAsyncProducer$4(DefaultProducerCache.java:329)
at org.apache.camel.processor.SendDynamicProcessor$1.done(SendDynamicProcessor.java:213)
at org.apache.camel.component.netty.http.NettyHttpProducer$NettyHttpProducerCallback.done(NettyHttpProducer.java:175)
at org.apache.camel.component.netty.NettyProducer$NettyProducerCallback.done(NettyProducer.java:571)
at org.apache.camel.component.netty.NettyCamelState.callbackDoneOnce(NettyCamelState.java:55)
at org.apache.camel.component.netty.handlers.ClientChannelHandler.channelRead0(ClientChannelHandler.java:223)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296)
at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1533)
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1282)
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1329)
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:501)
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:440)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.IllegalStateException: failed to create a child event loop
at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:88)
at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:58)
at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:47)
at io.netty.channel.MultithreadEventLoopGroup.<init>(MultithreadEventLoopGroup.java:59)
at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:86)
at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:81)
at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:68)
at org.apache.camel.component.netty.NettyWorkerPoolBuilder.build(NettyWorkerPoolBuilder.java:79)
at org.apache.camel.component.netty.NettyProducer.doStart(NettyProducer.java:137)
at org.apache.camel.support.service.BaseService.start(BaseService.java:115)
at org.apache.camel.support.service.ServiceHelper.startService(ServiceHelper.java:113)
at org.apache.camel.support.service.ServiceHelper.startService(ServiceHelper.java:130)
at org.apache.camel.component.rest.RestProducer.doStart(RestProducer.java:261)
at org.apache.camel.support.service.BaseService.start(BaseService.java:115)
at org.apache.camel.support.service.ServiceHelper.startService(ServiceHelper.java:113)
at org.apache.camel.impl.engine.AbstractCamelContext.internalAddService(AbstractCamelContext.java:1476)
at org.apache.camel.impl.engine.AbstractCamelContext.addService(AbstractCamelContext.java:1394)
at org.apache.camel.support.cache.ServicePool$SinglePool.acquire(ServicePool.java:210)
at org.apache.camel.support.cache.ServicePool$SinglePool.acquire(ServicePool.java:193)
at org.apache.camel.support.cache.ServicePool.acquire(ServicePool.java:109)
at org.apache.camel.support.cache.DefaultProducerCache.acquireProducer(DefaultProducerCache.java:126)
... 53 more
Caused by: io.netty.channel.ChannelException: failed to open a new selector
at io.netty.channel.nio.NioEventLoop.openSelector(NioEventLoop.java:175)
at io.netty.channel.nio.NioEventLoop.<init>(NioEventLoop.java:142)
at io.netty.channel.nio.NioEventLoopGroup.newChild(NioEventLoopGroup.java:146)
at io.netty.channel.nio.NioEventLoopGroup.newChild(NioEventLoopGroup.java:37)
at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:84)
... 73 more
Caused by: java.io.IOException: Unable to establish loopback connection
at java.base/sun.nio.ch.PipeImpl$Initializer.run(PipeImpl.java:94)
at java.base/sun.nio.ch.PipeImpl$Initializer.run(PipeImpl.java:61)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/sun.nio.ch.PipeImpl.<init>(PipeImpl.java:171)
at java.base/sun.nio.ch.SelectorProviderImpl.openPipe(SelectorProviderImpl.java:50)
at java.base/java.nio.channels.Pipe.open(Pipe.java:155)
at java.base/sun.nio.ch.WindowsSelectorImpl.<init>(WindowsSelectorImpl.java:142)
at java.base/sun.nio.ch.WindowsSelectorProvider.openSelector(WindowsSelectorProvider.java:44)
at io.netty.channel.nio.NioEventLoop.openSelector(NioEventLoop.java:173)
... 77 more
Caused by: java.net.SocketException: No buffer space available (maximum connections reached?): connect
at java.base/sun.nio.ch.Net.connect0(Native Method)
at java.base/sun.nio.ch.Net.connect(Net.java:483)
at java.base/sun.nio.ch.Net.connect(Net.java:472)
at java.base/sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:694)
at java.base/java.nio.channels.SocketChannel.open(SocketChannel.java:194)
at java.base/sun.nio.ch.PipeImpl$Initializer$LoopbackConnector.run(PipeImpl.java:127)
at java.base/sun.nio.ch.PipeImpl$Initializer.run(PipeImpl.java:76)
... 85 more
Later, I have added below code to close the connection after use
.process(exchange -> exchange.getMessage().setHeader(NettyConstants.NETTY_CLOSE_CHANNEL_WHEN_COMPLETE, true))
But still issue exists.
Since the above exception occurs only while triggering the external rest-api, Wanted to check on below 2 points,
Can someone please help on this. Thanks in advance.
Upvotes: 0
Views: 263