CODEWITHSUNDEEP
apache-camelconnectionjmsactivemq-classicconnection-pooling
Kim Zeevaarders
Kim Zeevaarders

Reputation: 805

ActiveMQ 5.10.2: [org.apache.activemq.broker.TransportConnection] TransportConnection (triggerStartAsyncNetworkBridgeCreation:Failed to add Connection

we're using camel 2.9.8 with ActiveMQ and recently upgraded ActiveMq from 5.5.1 to 5.10.2 (last JDK6 version because we are running on JBoss EAP 6). Our set up is a simple "network of brokers" (using discovery) with multiple nodes connecting over tcp:// and vm:// connectors.

It all works fine but after several days of 100% working I get sudden "fail to add connection" warnings in the logs. To be more precise:

2019-12-05 09:31:01,623 WARN  [org.apache.activemq.broker.TransportConnection] TransportConnection (triggerStartAsyncNetworkBridgeCreation: remoteBroker=tcp://nlheevpat01pha.x.global/10.32.50.163:61612@41557, localBroker= vm://nlheevpat02pha_prd_01#6262) Failed to add Connection nlheevpat02pha_prd_01->nlheevpat01pha_prd_01-55572-1575391926556-6265:1

We never got the warning in the old 5.5.1 version. The activeMQ queues still functions as normally and all expected conections are present under the "connections" tab of the activeMQ web admin console.

Our config looks as follows

<bean id="placeholderConfig"
    class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer" />

<!-- shutdown hook is disabled as RAR classloader may be gone at shutdown -->
<amq:broker xmlns="http://activemq.apache.org/schema/core"
    brokerName="${activemq.broker.name}"
    useJmx="true"
    useShutdownHook="false"
    networkConnectorStartAsync="true"
    dataDirectory="${activemq.data.dir}">

    <amq:destinationPolicy>
        <amq:policyMap>
            <amq:policyEntries>
                <amq:policyEntry queue=">" producerFlowControl="true" memoryLimit="500mb">
                    <amq:deadLetterStrategy>
                        <amq:individualDeadLetterStrategy queuePrefix="DLQ." />
                    </amq:deadLetterStrategy>
                </amq:policyEntry>
                <amq:policyEntry topic=">" producerFlowControl="true" memoryLimit="300mb">
                    <amq:deadLetterStrategy>
                        <amq:individualDeadLetterStrategy queuePrefix="DLQ." />
                    </amq:deadLetterStrategy>
                </amq:policyEntry>


                <amq:policyEntry queue="Service.Request" producerFlowControl="true" memoryLimit="3 gb">
                    <amq:dispatchPolicy>
                        <amq:roundRobinDispatchPolicy />
                    </amq:dispatchPolicy>
                </amq:policyEntry>

            </amq:policyEntries>

        </amq:policyMap>
    </amq:destinationPolicy>

    <amq:managementContext>
        <!-- use appserver provided context instead of creating one, for jboss use: -Djboss.platform.mbeanserver -->
        <amq:managementContext createConnector="false" />
    </amq:managementContext>

    <amq:networkConnectors>
        <amq:networkConnector
            uri="multicast://${activemq.network.multicast.address}:${activemq.network.multicast.port}?group=${activemq.network.multicast.group}"
            userName="${activemq.auth.system.username}"
            password="${activemq.auth.system.password}"
            conduitSubscriptions="false"
            decreaseNetworkConsumerPriority="true"
        />
    </amq:networkConnectors>

    <amq:persistenceAdapter>
        <amq:kahaDB
            directory="${activemq.data.dir}/kahadb"
            checkForCorruptJournalFiles="true"
            checksumJournalFiles="true"
        />
    </amq:persistenceAdapter>

    <amq:plugins>
        <amq:simpleAuthenticationPlugin>
            <amq:users>
                <amq:authenticationUser
                    username="${activemq.auth.system.username}"
                    password="${activemq.auth.system.password}"
                    groups="users"
                />
            </amq:users>
        </amq:simpleAuthenticationPlugin>

        <amq:authorizationPlugin>
            <amq:map>
                <amq:authorizationMap>
                    <amq:authorizationEntries>
                        <amq:authorizationEntry  queue=">" read="users" write="users" admin="users" />
                        <amq:authorizationEntry  topic=">" read="users" write="users" admin="users" />
                    </amq:authorizationEntries>
                </amq:authorizationMap>
            </amq:map>
        </amq:authorizationPlugin>
    </amq:plugins>

    <amq:systemUsage>
        <amq:systemUsage>
            <amq:memoryUsage>
                <amq:memoryUsage limit="3 gb" />
            </amq:memoryUsage>
            <amq:storeUsage>
                <amq:storeUsage limit="10 gb" />
            </amq:storeUsage>
            <amq:tempUsage>
                <amq:tempUsage limit="2 gb" />
            </amq:tempUsage>
        </amq:systemUsage>
    </amq:systemUsage>

    <amq:transportConnectors>
        <amq:transportConnector uri="tcp://${activemq.transport.host}:${activemq.transport.port}" discoveryUri="multicast://${activemq.network.multicast.address}:${activemq.network.multicast.port}?group=${activemq.network.multicast.group}" />
        <amq:transportConnector uri="vm://${activemq.broker.name}" />
    </amq:transportConnectors>

</amq:broker>

Does anybody of you have some pointers as to:

Hope to hear from you,

Kim

Upvotes: 2

Views: 559

Answers (1)

Ortomala Lokni
Ortomala Lokni

Reputation: 62733

Looking at the source code of ActiveMQ 5.10.2, there is a single occurence of the the string Failed to add Connection in TransportConnection.java:

try {
    broker.addConnection(context, info);
} catch (Exception e) {
    synchronized (brokerConnectionStates) {
        brokerConnectionStates.remove(info.getConnectionId());
    }
    unregisterConnectionState(info.getConnectionId());
    LOG.warn("Failed to add Connection {}", info.getConnectionId(), e);
    if (e instanceof SecurityException) {
        // close this down - in case the peer of this transport doesn't play nice
        delayedStop(2000, "Failed with SecurityException: " + e.getLocalizedMessage(), e);
    }
    throw e;
}

Your warning is caused by an exception, and the attempted connection is unregistered. At this point, I would inspect configuration files and disable all unneeded services and ports.

Can you share your configuration files ?

Upvotes: 2

Related Questions