CODEWITHSUNDEEP
spring-bootspring-integration
Atif Siddiqui
Atif Siddiqui

Reputation: 53

Spring Boot IMAP Channel Adapter is losing emails while processing a large number of inbound emails

I have a working Spring boot application which is processing emails from a dedicated mailbox. Everything works well in the test environment when the number of emails is limited. In Production environment, these emails are getting generated by a scheduled job and sometimes there are more than 10000 emails in one batch. Each email has some big attachments which the application is parsing and storing. This is where the application is slowing down badly and in the worst case scenarios losing the emails as well. Looks like a case of thread starvation as the process is resource intensive. Is there a way to optimize this flow?

My email config is as below. Service Activator snippet is from another service to handle the email message.

    @Bean
    public QueueChannel emailReceiveChannel() {
        return new QueueChannel(100);
    }

    @Bean(name = "emailPoller")
    public PollerMetadata poller() {
        PollerMetadata poller = new PollerMetadata();
        poller.setReceiveTimeout(0);
        poller.setTrigger(new PeriodicTrigger(2000));
        poller.setMaxMessagesPerPoll(100);
        poller.setTaskExecutor(taskExecutor());
        
        return poller;
    }
    
    @Bean(name = "emailTaskExecutor")
    public AsyncTaskExecutor taskExecutor() {
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(10);
        executor.setMaxPoolSize(1000);
        executor.setQueueCapacity(100);
        
        return executor;
    }

    @Bean
    public ImapIdleChannelAdapter mailAdapter() {
        
        ImapIdleChannelAdapter imapAdapter = new ImapIdleChannelAdapter(imapMailReceiver());
        imapAdapter.setAutoStartup(true);
        imapAdapter.setOutputChannel(emailReceiveChannel());
        
        return imapAdapter;
    }

    @Bean
    public ImapMailReceiver imapMailReceiver() {

        StringBuilder impaURI = new StringBuilder();
        impaURI.append(MAIL_PROTOCOL).append("://").append(MAIL_USERNAME).append(":").append(MAIL_PASSWORD)
        .append("@").append(MAIL_HOST).append(":").append(MAIL_PORT).append("/").append(MAIL_FOLDER);
        
        logger.info("IMAP URL = " + impaURI.toString());
        
        ImapMailReceiver mailReceiver = new ImapMailReceiver(impaURI.toString());
        mailReceiver.setShouldDeleteMessages(true);
        mailReceiver.setShouldMarkMessagesAsRead(true);
        mailReceiver.setJavaMailProperties(javaMailProperties());
        mailReceiver.setAutoCloseFolder(false);

        return mailReceiver;
    }
     
    @ServiceActivator(inputChannel = "emailReceiveChannel", poller = @Poller(value = "emailPoller"))
    public void handleMessage(final Message message) {
        
        logger.info("Email Message received on " + LocalDate.now());
        if (message != null && message instanceof MimeMessage) {
            /*
             * Delegate the message handling to another thread and release this one.
             */
            processEmailMessage(message);
        }
    }

    /*
     * Global Task scheduler
     */
    @Bean(name = "taskScheduler")
    public TaskScheduler taskScheduler() {
        ThreadPoolTaskScheduler scheduler = new ThreadPoolTaskScheduler();
        scheduler.setPoolSize(100);
        return scheduler;
    }

Upvotes: 2

Views: 226

Answers (1)

Artem Bilan
Artem Bilan

Reputation: 121272

You probably need to consider to increase TaskScheduler thread pool: https://docs.spring.io/spring-integration/docs/current/reference/html/configuration.html#namespace-taskscheduler.

You also may consider do not use that emailReceiveChannel as a queue. The ImapIdleChannelAdapter is async by nature and already does its logic within scheduled thread.

Upvotes: 1

Related Questions