CODEWITHSUNDEEP
springjdbcdb2webspherequartz-scheduler
Suchak Jani
Suchak Jani

Reputation: 157

quartz scheduler cannot acquire trigger

I am working with a client and getting issues where jobs are not triggered from time to time.

Here is an example of an error, this is happening frequently and also causing jobs not to be triggered.

Mon, 15 May 2017 04:54:03-04:00 2017-05-15T08:54:03.0000405Z [xyz_QuartzSchedulerThread] ERROR c.v.i.c.s.s.s.QuartzMonitorImpl - Failure encountered by the Quartz Scheduler, An error occurred while scanning for the next triggers to fire.
org.quartz.JobPersistenceException: Couldn't acquire next trigger: The current transaction has been rolled back because of a deadlock or timeout.  Reason code "2".. SQLCODE=-911, SQLSTATE=40001, DRIVER=3.69.49
    at org.quartz.impl.jdbcjobstore.JobStoreSupport.acquireNextTrigger(JobStoreSupport.java:2848) ~[quartz-2.2.1.jar:na]
    at org.quartz.impl.jdbcjobstore.JobStoreSupport$40.execute(JobStoreSupport.java:2759) ~[quartz-2.2.1.jar:na]
    at org.quartz.impl.jdbcjobstore.JobStoreSupport$40.execute(JobStoreSupport.java:2757) ~[quartz-2.2.1.jar:na]
    at org.quartz.impl.jdbcjobstore.JobStoreSupport.executeInNonManagedTXLock(JobStoreSupport.java:3787) ~[quartz-2.2.1.jar:na]
    at org.quartz.impl.jdbcjobstore.JobStoreSupport.acquireNextTriggers(JobStoreSupport.java:2756) ~[quartz-2.2.1.jar:na]
    at org.quartz.core.QuartzSchedulerThread.run(QuartzSchedulerThread.java:272) ~[quartz-2.2.1.jar:na]
Caused by: com.ibm.db2.jcc.am.SqlTransactionRollbackException: The current transaction has been rolled back because of a deadlock or timeout.  Reason code "2".. SQLCODE=-911, SQLSTATE=40001, DRIVER=3.69.49
    at com.ibm.db2.jcc.am.gd.a(Unknown Source) ~[db2jcc.jar:na]
    at com.ibm.db2.jcc.am.gd.a(Unknown Source) ~[db2jcc.jar:na]
    at com.ibm.db2.jcc.am.gd.a(Unknown Source) ~[db2jcc.jar:na]
    at com.ibm.db2.jcc.am.yo.c(Unknown Source) ~[db2jcc.jar:na]
    at com.ibm.db2.jcc.t4.bb.t(Unknown Source) ~[db2jcc.jar:na]
    at com.ibm.db2.jcc.t4.bb.l(Unknown Source) ~[db2jcc.jar:na]
    at com.ibm.db2.jcc.t4.bb.a(Unknown Source) ~[db2jcc.jar:na]
    at com.ibm.db2.jcc.t4.p.a(Unknown Source) ~[db2jcc.jar:na]
    at com.ibm.db2.jcc.t4.wb.b(Unknown Source) ~[db2jcc.jar:na]
    at com.ibm.db2.jcc.am.zo.qc(Unknown Source) ~[db2jcc.jar:na]
    at com.ibm.db2.jcc.am.zo.b(Unknown Source) ~[db2jcc.jar:na]
    at com.ibm.db2.jcc.am.zo.ic(Unknown Source) ~[db2jcc.jar:na]
    at com.ibm.db2.jcc.am.zo.executeUpdate(Unknown Source) ~[db2jcc.jar:na]
    at com.ibm.ws.rsadapter.jdbc.WSJdbcPreparedStatement.pmiExecuteUpdate(WSJdbcPreparedStatement.java:1187) ~[com.ibm.ws.runtime.jar:na]
    at com.ibm.ws.rsadapter.jdbc.WSJdbcPreparedStatement.executeUpdate(WSJdbcPreparedStatement.java:804) ~[com.ibm.ws.runtime.jar:na]
    at org.quartz.impl.jdbcjobstore.StdJDBCDelegate.updateTriggerStateFromOtherState(StdJDBCDelegate.java:1439) ~[quartz-2.2.1.jar:na]
    at org.quartz.impl.jdbcjobstore.JobStoreSupport.acquireNextTrigger(JobStoreSupport.java:2827) ~[quartz-2.2.1.jar:na]
    ... 5 common frames omitted
Mon, 15 May 2017 04:54:03-04:00 2017-05-15T08:54:03.0000408Z [settlementScheduler_QuartzSchedulerThread] ERROR org.quartz.core.ErrorLogger - An error occurred while scanning for the next triggers to fire.
org.quartz.JobPersistenceException: Couldn't acquire next trigger: The current transaction has been rolled back because of a deadlock or timeout.  Reason code "2".. SQLCODE=-911, SQLSTATE=40001, DRIVER=3.69.49
    at org.quartz.impl.jdbcjobstore.JobStoreSupport.acquireNextTrigger(JobStoreSupport.java:2848) ~[quartz-2.2.1.jar:na]
    at org.quartz.impl.jdbcjobstore.JobStoreSupport$40.execute(JobStoreSupport.java:2759) ~[quartz-2.2.1.jar:na]
    at org.quartz.impl.jdbcjobstore.JobStoreSupport$40.execute(JobStoreSupport.java:2757) ~[quartz-2.2.1.jar:na]
    at org.quartz.impl.jdbcjobstore.JobStoreSupport.executeInNonManagedTXLock(JobStoreSupport.java:3787) ~[quartz-2.2.1.jar:na]
    at org.quartz.impl.jdbcjobstore.JobStoreSupport.acquireNextTriggers(JobStoreSupport.java:2756) ~[quartz-2.2.1.jar:na]
    at org.quartz.core.QuartzSchedulerThread.run(QuartzSchedulerThread.java:272) ~[quartz-2.2.1.jar:na]
Caused by: com.ibm.db2.jcc.am.SqlTransactionRollbackException: The current transaction has been rolled back because of a deadlock or timeout.  Reason code "2".. SQLCODE=-911, SQLSTATE=40001, DRIVER=3.69.49
    at com.ibm.db2.jcc.am.gd.a(Unknown Source) ~[db2jcc.jar:na]
    at com.ibm.db2.jcc.am.gd.a(Unknown Source) ~[db2jcc.jar:na]
    at com.ibm.db2.jcc.am.gd.a(Unknown Source) ~[db2jcc.jar:na]
    at com.ibm.db2.jcc.am.yo.c(Unknown Source) ~[db2jcc.jar:na]
    at com.ibm.db2.jcc.t4.bb.t(Unknown Source) ~[db2jcc.jar:na]
    at com.ibm.db2.jcc.t4.bb.l(Unknown Source) ~[db2jcc.jar:na]
    at com.ibm.db2.jcc.t4.bb.a(Unknown Source) ~[db2jcc.jar:na]
    at com.ibm.db2.jcc.t4.p.a(Unknown Source) ~[db2jcc.jar:na]
    at com.ibm.db2.jcc.t4.wb.b(Unknown Source) ~[db2jcc.jar:na]
    at com.ibm.db2.jcc.am.zo.qc(Unknown Source) ~[db2jcc.jar:na]
    at com.ibm.db2.jcc.am.zo.b(Unknown Source) ~[db2jcc.jar:na]
    at com.ibm.db2.jcc.am.zo.ic(Unknown Source) ~[db2jcc.jar:na]
    at com.ibm.db2.jcc.am.zo.executeUpdate(Unknown Source) ~[db2jcc.jar:na]
    at com.ibm.ws.rsadapter.jdbc.WSJdbcPreparedStatement.pmiExecuteUpdate(WSJdbcPreparedStatement.java:1187) ~[com.ibm.ws.runtime.jar:na]
    at com.ibm.ws.rsadapter.jdbc.WSJdbcPreparedStatement.executeUpdate(WSJdbcPreparedStatement.java:804) ~[com.ibm.ws.runtime.jar:na]
    at org.quartz.impl.jdbcjobstore.StdJDBCDelegate.updateTriggerStateFromOtherState(StdJDBCDelegate.java:1439) ~[quartz-2.2.1.jar:na]
    at org.quartz.impl.jdbcjobstore.JobStoreSupport.acquireNextTrigger(JobStoreSupport.java:2827) ~[quartz-2.2.1.jar:na]
    ... 5 common frames omitted

Details of environment

Would appreciate if someone has already faced the issue and can shed some light on this

Upvotes: 0

Views: 7368

Answers (1)

pufface
pufface

Reputation: 273

I had similar problem. Quartz complained about deadlock or timeout - similar to your stacktrace. In my case, exception message was bit misleading, because it actually meant no connection to database at all.

  • Make sure that you meet required criteria for using JobStoreCMT according to quartz documentaiton:

    ...A JTA transaction must be in progress before attempt to schedule (or unschedule) jobs/triggers. This allows the “work” of scheduling to be part of the applications “larger” transaction...

  • Be aware - your scheduled jobs run in unmanaged threads (outside Websphere) as long as using SimpleThreadPool. Unmanaged thread does not have access to server context (including data sources, jndi , etc.)

  • Consider to use JobStoreTX and run jobs inside managed threads provided by WorkManagers, see discussion and my answer with github-example.

Upvotes: 0

Related Questions