Reputation: 71
We are running WildFly 12 with JDK 1.8 version. We faced the portal down / slowness issue in our web application due to high CPU usage (90% above) of java.exe. Then, we have taken a thread dump using JStack and most of the high CPU threads are pointing to "org.xnio.nio"
Portal is running in HTTPS
CPU: 8 core
Total RAM: 24 GB
Allocated JVM memory: From 1 GB (min) to 5 GB (max)
Occupaid JVM memory when taken the thread dump: 4 GB
DB is in separate machine
Below are the thread dump results (High CPU threads):
"default I/O-1" #95 prio=5 os_prio=0 tid=0x00000000260bf800 nid=0x1988 runnable [0x000000002e49f000]
java.lang.Thread.State: RUNNABLE
at io.undertow.server.protocol.http.HttpReadListener.handleEventWithNoRunningRequest(HttpReadListener.java:147)
at io.undertow.server.protocol.http.HttpReadListener.handleEvent(HttpReadListener.java:136)
at io.undertow.server.protocol.http.HttpReadListener.handleEvent(HttpReadListener.java:59)
at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
at org.xnio.conduits.ReadReadyHandler$ChannelListenerHandler.readReady(ReadReadyHandler.java:66)
at io.undertow.conduits.ReadTimeoutStreamSourceConduit$2.readReady(ReadTimeoutStreamSourceConduit.java:89)
at io.undertow.protocols.ssl.SslConduit$SslReadReadyHandler.readReady(SslConduit.java:1145)
at org.xnio.nio.NioSocketConduit.handleReady(NioSocketConduit.java:89)
at org.xnio.nio.WorkerThread.run(WorkerThread.java:591)
"default I/O-13" #108 prio=5 os_prio=0 tid=0x0000000022186000 nid=0x2310 runnable [0x000000002f09f000]
java.lang.Thread.State: RUNNABLE
at java.lang.Object.notifyAll(Native Method)
at sun.nio.ch.WindowsSelectorImpl$StartLock.startThreads(WindowsSelectorImpl.java:189)
- locked <0x00000006618ae440> (a sun.nio.ch.WindowsSelectorImpl$StartLock)
at sun.nio.ch.WindowsSelectorImpl$StartLock.access$300(WindowsSelectorImpl.java:181)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:153)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x000000066508ea18> (a sun.nio.ch.Util$3)
- locked <0x000000066508ea08> (a java.util.Collections$UnmodifiableSet)
- locked <0x0000000665082b98> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.xnio.nio.WorkerThread.run(WorkerThread.java:551)
"default I/O-13" #108 prio=5 os_prio=0 tid=0x0000000022186000 nid=0x2310 runnable [0x000000002f09e000]
java.lang.Thread.State: RUNNABLE
at java.util.AbstractCollection.toArray(AbstractCollection.java:183)
at sun.nio.ch.Util$3.toArray(Util.java:322)
at org.xnio.nio.WorkerThread.run(WorkerThread.java:570)
- locked <0x000000066508ea18> (a sun.nio.ch.Util$3)
- locked <0x0000000665082b98> (a sun.nio.ch.WindowsSelectorImpl)
"default I/O-5" #100 prio=5 os_prio=0 tid=0x00000000260c0000 nid=0x858 runnable [0x000000002e89e000]
java.lang.Thread.State: RUNNABLE
at org.xnio.nio.WorkerThread.run(WorkerThread.java:480)
The issue is occuring (2 - 5) days once.
Thanks in advance
Upvotes: 1
Views: 904