CODEWITHSUNDEEP
spring-bootdatadogmicrometer
tofagerl
tofagerl

Reputation: 95

Why won't Micrometer stop sending data to datadog and just close already?

I have a Spring Boot app which due to weird restrictions needs to run once every three hours, and won't work with Quartz, so I've been running it once every three hours from OS cron and it quits when it's done. After adding micrometer-registry-datadog (and spring-legacy) however, it never quits, it just sends metrics every 20 seconds or whatever the default period is, even after calling registry.close().

Am I doomed like the dutchman to sail the seas of processing forever, or is there an obvious error I have made?

Code: It reaches SpringApplication.exit(ctx), but it does not actually exit cleanly. (service is a TimedExecutorService.)

        public void close() throws InterruptedException {
            service.shutdown();
            service.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS);
            meterRegistry.close();
            SpringApplication.exit(ctx);
    }

Upvotes: 0

Views: 1282

Answers (1)

checketts
checketts

Reputation: 14983

This sounds like a bug. It is possible the Datadog exporter is running in a non-daemon thread. The JVM views non-daemon threads as application critical work.

So essentially the JVM thinks it shouldn't shutdown until the non-daemon thread finishes. In the case of the Datadog exporter thread, that probably won't happen.

To verify there are non-daemon threads, use jstack to generate a thread dump. (command: jstack <pid>) or dump all threads in your close method:

ThreadMXBean threadMxBean = ManagementFactory.getThreadMXBean();
for (ThreadInfo ti : threadMxBean.dumpAllThreads(true, true)) {
  System.out.print(ti.toString());
}

An example thread dump output is below. Notice the word 'daemon' on the first line:

"pool-1-thread-1" #13 prio=5 os_prio=31 tid=0x00007fe885aa5000 nid=0xa907 waiting on condition [0x000070000d67b000]
   java.lang.Thread.State: WAITING (parking)
    at sun.misc.Unsafe.park(Native Method)
    - parking to wait for  <0x00000006c07e9720> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

Upvotes: 0

Related Questions