CODEWITHSUNDEEP
activemq-artemis
recvfrom_ro
recvfrom_ro

Reputation: 107

Artemis shutting down with AMQ222010: Critical IO Error, shutting down the server [ask for clarification]

Does this fix mentioned here fix an error that was introduced in 2.16 or does it fix a long existing problem?

For a long time we have periodical ActiveMQ Artemis stoppages with similar threaddump + shutdown combo starting with

09:24:10,086 WARN  [org.apache.activemq.artemis.core.server] AMQ222010: Critical IO Error, shutting down the server. file=AIOSequentialFile:/home/jms-artemis/jms-broker/./data/journal/activemq-data-267599.amq, message=Timeout on close: java.io.IOException: Timeout on close
        at org.apache.activemq.artemis.core.io.aio.AIOSequentialFile.close(AIOSequentialFile.java:126) [artemis-journal-2.4.0.jar:2.4.0]
        at org.apache.activemq.artemis.core.io.aio.AIOSequentialFile.close(AIOSequentialFile.java:103) [artemis-journal-2.4.0.jar:2.4.0]
        at org.apache.activemq.artemis.core.journal.impl.JournalFilesRepository.closeFile(JournalFilesRepository.java:481) [artemis-journal-2.4.0.jar:2.4.0]
        at org.apache.activemq.artemis.core.journal.impl.JournalImpl.moveNextFile(JournalImpl.java:3021) [artemis-journal-2.4.0.jar:2.4.0]
        at org.apache.activemq.artemis.core.journal.impl.JournalImpl.switchFileIfNecessary(JournalImpl.java:2960) [artemis-journal-2.4.0.jar:2.4.0]
        at org.apache.activemq.artemis.core.journal.impl.JournalImpl.appendRecord(JournalImpl.java:2679) [artemis-journal-2.4.0.jar:2.4.0]
        at org.apache.activemq.artemis.core.journal.impl.JournalImpl.access$200(JournalImpl.java:90) [artemis-journal-2.4.0.jar:2.4.0]
        at org.apache.activemq.artemis.core.journal.impl.JournalImpl$6.run(JournalImpl.java:1063) [artemis-journal-2.4.0.jar:2.4.0]
        at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42) [artemis-commons-2.4.0.jar:2.4.0]
        at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31) [artemis-commons-2.4.0.jar:2.4.0]
        at org.apache.activemq.artemis.utils.actors.ProcessorBase$ExecutorTask.run(ProcessorBase.java:53) [artemis-commons-2.4.0.jar:2.4.0]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_211]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_211]
        at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_211]

09:24:10,101 WARN  [org.apache.activemq.artemis.journal] waiting pending callbacks on activemq-data-267599.amq from 10 seconds!
<...>

And for a long time the only similar problem-resolution was this post which states the solution is:

Check disk storage health on a hardware level in order to avoid such issues in the future.

That was not our case of course, but we could not figure out what the core of the problem was, so we just adjusted to the periodical downtime. But few days ago noticed a very similar question where it was advised to move to 2.17.0. However, the referenced issue states that it affects version 2.16/0.

We had no failback configuration, and we are using is 2.4.0.

The diff is really hard to get for external person, and the distance between 2.4.0 and 2.16.0 is far enough, so the question is does the stated 2.17.0 version fix some kind of similar but new problem introduced in 2.16, or the problem was long lurking and to overcome we should move from 2.4 to 2.17 and happiness will come?

Upvotes: 0

Views: 1440

Answers (1)

Justin Bertram
Justin Bertram

Reputation: 35008

The "Affects Version" on ARTEMIS-3084 is 2.16.0, but that's just the most recent version that exhibited the problem. I'm confident the issue was present in previous versions. If you're experiencing this issue I recommend you move to 2.17.0.

Upvotes: 1

Related Questions