CODEWITHSUNDEEP
springmultithreadingspring-integrationjakarta-mailimap
Sundararaj Govindasamy
Sundararaj Govindasamy

Reputation: 8495

Spring IMAP Poller reads emails after some hours of delay

I am using Spring Integration to poll emails from 3 mailboxes using 3 adapters. I am using customSearchTermStatregy to read emails after 10 mins of receiving.

This application reads email and saving those email into local directory as .eml file.

Now the problem is, some email are read after 4-5 hours of delay by the poller.

I configured a thread pool as well, but I suspect spring poller is not getting thread allocated immediately to read email, hence the delay in reading.

Please also note, this application is running under tomcat along with other 10+ java applications running under same tomcat.

Can someone point out some probable reasons behind this 4-5 hours of delay?

Spring Integration.xml:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:mail="http://www.springframework.org/schema/integration/mail"
       xmlns:int="http://www.springframework.org/schema/integration"
       xmlns:util="http://www.springframework.org/schema/util"
       xmlns:task="http://www.springframework.org/schema/task"       
       xsi:schemaLocation="http://www.springframework.org/schema/beans
            http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
            http://www.springframework.org/schema/integration/mail
            http://www.springframework.org/schema/integration/mail/spring-integration-mail-5.2.xsd 
            http://www.springframework.org/schema/util 
            http://www.springframework.org/schema/util/spring-util-2.0.xsd
            http://www.springframework.org/schema/task 
            http://www.springframework.org/schema/task/spring-task-4.3.xsd">

    <util:properties id="javaMailProperties">
        <prop key="mail.imap.socketFactory.class">javax.net.ssl.SSLSocketFactory</prop>
        <prop key="mail.imap.socketFactory.fallback">false</prop>
        <prop key="mail.store.protocol">imaps</prop>
        <prop key="mail.debug">${imap.debug}</prop>
        <prop key="mail.imaps.partialfetch">false</prop>
        <prop key="mail.imaps.fetchsize">102400</prop>   <!-- 100KB, default is 16KB -->  
    </util:properties>
    
    <task:executor id="taskExecutor"  pool-size="15-50" queue-capacity="100"/>

    <mail:inbound-channel-adapter id="imapAdapter1" 
                                      store-uri="${imap.uri.mail}"                                     
                                      channel="recieveEmailChannel"                                          
                                      should-delete-messages="false"
                                      should-mark-messages-as-read="true"                                      
                                      auto-startup="true"
                                      simple-content="true"
                                      auto-close-folder="false"
                                      search-term-strategy="mailSearchTermStrategy"
                                      java-mail-properties="javaMailProperties">
       <!-- <int:poller fixed-delay="${imap.polling.interval}" time-unit="SECONDS"/> -->
       <int:poller cron="0 0/2 1-23 * * ?" max-messages-per-poll="1" task-executor="taskExecutor"/> <!-- every 2 mins, from 1AM to 11.59PM -->
    </mail:inbound-channel-adapter>
    
    <mail:inbound-channel-adapter id="imapAdapter2" 
                                      store-uri="${imap.uri.fax}"                                     
                                      channel="recieveEmailChannel"                                          
                                      should-delete-messages="false"
                                      should-mark-messages-as-read="true"                                      
                                      auto-startup="true"
                                      simple-content="true"
                                      auto-close-folder="false"
                                      search-term-strategy="faxSearchTermStrategy"
                                      java-mail-properties="javaMailProperties">
       <int:poller cron="0 0/2 1-23 * * ?" max-messages-per-poll="1" task-executor="taskExecutor"/> <!-- every 2 mins, from 1AM to 11.59PM -->
    </mail:inbound-channel-adapter>
    
    <mail:inbound-channel-adapter id="imapAdapter3" 
                                      store-uri="${imap.uri.scan}"                                     
                                      channel="recieveEmailChannel"                                          
                                      should-delete-messages="false"
                                      should-mark-messages-as-read="true"                                      
                                      auto-startup="true"
                                      simple-content="true"
                                      auto-close-folder="false"
                                      search-term-strategy="scanSearchTermStrategy"
                                      java-mail-properties="javaMailProperties">
      <int:poller cron="0 0/2 1-23 * * ?" max-messages-per-poll="1" task-executor="taskExecutor"/> <!-- every 2 mins, from 1AM to 11.59PM -->
     </mail:inbound-channel-adapter>

    <int:channel id="recieveEmailChannel">        
        <int:interceptors>
            <int:wire-tap channel="logger"/>
        </int:interceptors>
    </int:channel>

    <int:logging-channel-adapter id="logger" level="DEBUG"/>

    <int:service-activator input-channel="recieveEmailChannel" ref="emailReceiver" method="handleMessage"/>

  <!-- <bean id="emailReceiver" class="com.abc.xyz.receiver.EmailReceiver">
    </bean>  -->  
</beans>

CustomSearchTermStrategy:

@Service
public class CustomSearchTermStrategy implements SearchTermStrategy {
    
    @Value("${email.delay.mins}")
    int emailDelayMins; 
    
    Logger logger = LoggerFactory.getLogger(CustomSearchTermStrategy.class);
    
    @Override
    public SearchTerm generateSearchTerm(Flags supportedFlags, Folder folder) {
        
         SearchTerm customSentDateTerm = new SearchTerm(){           
         private static final long serialVersionUID = 4583016738325920713L;

            public boolean match(Message message) {
                 try {
                     ZoneId zoneId = ZoneId.of("America/Los_Angeles");                  
                     ZonedDateTime sendDtInPST =  message.getSentDate().toInstant().atZone(zoneId);              
                     ZonedDateTime currentZdt = ZonedDateTime.now(zoneId);                  
                     ZonedDateTime currentTimeMinus10Mins = currentZdt.minusMinutes(emailDelayMins);
                     Flags flags = message.getFlags();
 
                     if(currentTimeMinus10Mins.isAfter(sendDtInPST)) {                       
                        if(!flags.contains(Flags.Flag.SEEN)) {
                         logger.info("CurrentTimeMinus"+emailDelayMins+"MinsInPST is AFTER sendDtInPST, so this email picked for read, subject:{}", message.getSubject());
                         return true;   
                        }
                     }   
                 } catch (Exception ex) {
                     ex.printStackTrace();
                  }
                 return false;
             }
         };
        return customSentDateTerm;
    }
}

Upvotes: 0

Views: 658

Answers (2)

Arunkumar
Arunkumar

Reputation: 41

In our case, the delay in polling a large email from the IMAP server was because we used default values for properties mail.imap.fetchsize=16kb and mail.imap.partialfetch=true.

This made the email to be fetched in sizes of 16kb which resulted in long hours for polling. So we set the mail.imap.partialfetch=false to ensure that the mail gets fetched at a stretch

Refer this https://javaee.github.io/javamail/docs/api/com/sun/mail/imap/package-summary.html for further property setting in IMAP

Upvotes: 2

Artem Bilan
Artem Bilan

Reputation: 121177

At the moment you show 3 pollers which are going to be run at the same time. By default all those pollers are triggered by the TaskScheduler (don't mix it with TaskExecutor), which is based on 10 threads pool. So, even right now I see 3 concurrent threads taken by your email channel adapters. It might not be a surprise that you have some other pollers in your your applications. And all of them share the same TaskScheduler.

You may revise all your pollers to shift the real work into particular executors, or you may consider to increase the TaskScheduler thread pool: https://docs.spring.io/spring-integration/docs/current/reference/html/configuration.html#namespace-taskscheduler

You also can do a thread dump to analyze which threads are busy and for what.

Upvotes: 1

Related Questions