CODEWITHSUNDEEP
rabbitmqspring-amqp
Pand005
Pand005

Reputation: 1175

Why spring-amqp consumer performance is very slow?

I have started producer and consumer concurrently. After 6 hours producer produced around 6 crores messages into queue and stopped producer after 6 hours but consumer is running continuously, even after running 18 hours still 4 crores messages are in queue. Could any one please let me know why consumer performance is very slow?

Thanks in advance!

@Bean
    public SimpleMessageListenerContainer listenerContainer() {
        SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
        container.setConnectionFactory(connectionFactory());
        container.setQueueNames(this.queueName);
        container.setMessageListener(new MessageListenerAdapter(new TestMessageHandler(), new JsonMessageConverter()));
        return container;
    }
@Bean
    public ConnectionFactory connectionFactory() {
        CachingConnectionFactory connectionFactory = new CachingConnectionFactory(
                "localhost");
        connectionFactory.setUsername("guest");
        connectionFactory.setPassword("guest");
        return connectionFactory;
    }

    @Bean
    public RabbitTemplate rabbitTemplate() {
        RabbitTemplate template = new RabbitTemplate(connectionFactory());
        template.setMessageConverter(new JsonMessageConverter());
        template.setRoutingKey(this.queueName);
        template.setQueue(this.queueName);
        return template;
    }

    public class TestMessageHandler  {
           // receive messages
        public void handleMessage(MessageBeanTest msgBean) {
                   //  Storing bean data into CSV file
             }
    }

Upvotes: 3

Views: 4758

Answers (2)

Gary Russell
Gary Russell

Reputation: 174584

According to WikiPedia, crore == 10,000,000 so you mean 60 million.

The container can only process messages as fast as your listener does - you need to analyze what you are doing with each message.

You also need to experiment with the container concurrency settings (concurrentConsumers), prefetch, etc, to obtain the optimum performance, but it still ends up being your listener that takes the majority of the processing time; the container has very little overhead. Increasing the concurrency won't help if your listener is not well constructed.

If you are using transactions, that will significantly slow down consumption.

Try using a listener that does nothing with the message.

Finally, you should always show configuration when asking questions like this.

Upvotes: 2

jmathewt
jmathewt

Reputation: 947

As per Gary's suggestion you can set them as follows. Check out @RabbitListener

@Bean
public SimpleRabbitListenerContainerFactory listenerContainer(     {
    SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
    factory.setConnectionFactory(baseConfig.connectionFactory());
    factory.setConcurrentConsumers(7); // choose a value
    factory.setPrefetchCount(1); // how many messages per consumer at a time
    factory.setMaxConcurrentConsumers(10); // choose a value
    factory.setDefaultRequeueRejected(false); // if you want to deadletter
    return factory;
}

Upvotes: 4

Related Questions