Abhinavece
Abhinavece

Reputation: 201

Chronicle Queue Heap memory issue

As per the heap dump, it seems there are huge number of StoreTailer instances getting created and it's occupying 25% of memory. can you help me understand what I am doing wrong?

    3,14,071 instances of "net.openhft.chronicle.queue.impl.single.StoreTailer", \n loaded by "org.springframework.boot.loader.LaunchedURLClassLoader @ 0x5339526d0" occupy 37,01,94,288 (24.97%) bytes. These instances are referenced from one instance of "java.lang.Object[]", loaded by "<system class loader>"
    
    3,12,605 instances of "net.openhft.chronicle.queue.impl.single.SCQIndexing", loaded by "org.springframework.boot.loader.LaunchedURLClassLoader @ 0x5339526d0" occupy 26,75,84,536 (18.05%) bytes. These instances are referenced from one instance of "java.lang.Object[]", loaded by "<system class loader>"
    
    
    9,37,551 instances of "net.openhft.chronicle.bytes.internal.ChunkedMappedBytes", loaded by "org.springframework.boot.loader.LaunchedURLClassLoader @ 0x5339526d0" occupy 24,75,13,464 (16.70%) bytes. These instances are referenced from one instance of "java.lang.Object[]", loaded by "<system class loader>"

Here is the implementation:

 while (!batchToSend.isFull()) {
    long endIndex = queue.createTailer().toEnd().index();
    try (DocumentContext dc = readTailer.readingDocument()) {
      if (!dc.isPresent()) {
        sampler.sampled(() -> log.info("Reached end of queue"));
        long readIndex = readTailer.index();
        if (readIndex < endIndex) {
          readTailer.moveToIndex(endIndex);
          fileDeletionManager.setSentIndex(endIndex);
          log.warn(
              "Observed readTailer not at end with no document context. Moved from {} to {}", readIndex, endIndex);
        }
        break;
      }
      try {
        verify(dc.wire() != null, "Null wire with document context present");
        byte[] bytes = requireNonNull(dc.wire()).read().bytes();
        if (bytes != null) {
          PublishMessage message = PublishMessage.parseFrom(bytes);
          batchToSend.add(message);
        } else {
          // could happen in case of an error during append with document context open.
          log.warn("Read NULL message. Skipping");
        }
      } catch (Exception e) {
        log.error("Exception while parsing message", e);
      }
    }
  }

Upvotes: 0

Views: 77

Answers (0)

Related Questions