CODEWITHSUNDEEP
kdb+
Utsav
Utsav

Reputation: 5918

Difference in count of messages between RDB and TP while replaying the logfile from TP in kdb

Reference - https://code.kx.com/q/wp/rt-tick/#tickerplant-log-replay

The second argument to .u.rep is simpler and would look like:


q)y
11995
`:C:/OnDiskDB/sym2014.08.23
In other words, y is a pair where the last element is the TP logfile and the first element is the number of messages written to this logfile so far.     
This is the number of messages which the RDB will replay. Given the single-threaded nature of this process, the RDB will neither miss nor duplicate any of these messages.

In case RDB is down at nearly end of the day and we restart it, RDB subscribes to TP using .u.rep and get the count of messages in TP log file and the location of log file as

q) y
10000000 /-  10 million
`:/location

Say, it took 5 mins for RDB to replay 10 million messages from the log file but in those 5 mins TP receives 1000 messages from FeedHandler.
What will happen to those 1000 messages, where will these 1000 messages live meanwhile and how will RDB receive messages after 10 million messages?

Upvotes: 0

Views: 173

Answers (1)

terrylynch
terrylynch

Reputation: 13657

While the RDB is replaying the 1000 message get stored in the output queue in the tickerplant process. You can see the output queue with .z.W.

Once the RDB is finished replaying the 10mil message it will finally start to receive the records from the TPs output queue until fully drained by which time the RDB is fully caught up and in sync with the tickerplant

Upvotes: 2

Related Questions