CODEWITHSUNDEEP
chronicle-queuelinkedblockingqueue
Jack Copper
Jack Copper

Reputation: 33

Does a ChronicleQueue block on reads after a tailer has retrieved the most recently written excerpt?

I too am beginning to look at ChronicleQueue.

From the documentation:

Reading the queue follows the same pattern as writing, except there is a possibility there is not a message when you attempt to read it

I am trying to understand what happens when there is no message. The beauty of a LinkedBlockingQueue is that it just works, with little CPU load, when one thread is putting items into a queue (say, from a stream) and another thread is consuming items (yes, removing elements as they arrive and processing them in small groups). I would like to take advantage of the mapped files for history for raw items from the stream and the performance of a ChronicleQueue, but it seems that to replicate the behavior of a LinkedBlockingQueue my code will have to keep track of excerpt indices to know what excerpts have been processed - either when they arrive (placed in queue), or through some machinations using replay in the code that actually process the content of an excerpt. That said, I suspect I am missing something fundamental in my understanding of appender/tailer behavior. Any insights would be appreciated.

Upvotes: 0

Views: 375

Answers (1)

Dmitry Pisklov
Dmitry Pisklov

Reputation: 1206

Chronicle Queue is designed for low-latency, single-threaded environments and hence it never blocks on any operations.

Therefore, the usual pattern of reading from the queue involves busy-spinning (or various types of pausers, the simplest being Thread.yield()), e.g.:

while (true) {
    try (DocumentContext dc = tailer.readingDocument()) {
        // false here means there's no data to read, so we just keep waiting
        if (dc.isPresent()) {
             Bytes bytes = dc.wire().bytes();
            // read from bytes
        }
    }
}

Unless you want to return to certain items in your queue later, you don't need to bother with indices or anything of this sort.

NB: busy-spinning is opposed to blocking and it WILL consume 100% of CPU, and that's a deliberate trade-off for low-latency solutions.

Upvotes: 1

Related Questions