CODEWITHSUNDEEP
kdb+
Piotr Borowczynski
Piotr Borowczynski

Reputation: 195

kdb+ replication of RDB and HDB

I am trying to figure out how to implement/configure replication of RDB and HDB of kdb+ How to configure RDB and HDB to have two instances with the same data on different hosts?

Upvotes: 2

Views: 748

Answers (2)

Rahul
Rahul

Reputation: 3969

Couple of other approaches that you could try. Since KDB does not have a replication management system there are chances of data loss or data out of order on replicas. So you will have to think how to handle those situations.

One easy solution to handle those situations is to maintain sequence number for each update. In case replica detects missing sequence or out of order sequence, it can ask primary to send the data for those missing sequences.

  1. Chained ticker plant: You could try setup similar to chained tickerplant style. Secondary ticker plant subscribes to primary and then secondary setup run like a normal setup.

  2. Secondary RDB/TP subscribes to Primary RDB: Secondary RDB or TP gets updates from primary RDB. Secondary HDB would work normal.

  3. Separate process for replication management: If you want multiple replicas then it would increase load on primary if all of them connect to primary. Also, managing data loss scenarios would be difficult.

    Instead, you could create a separate process and all replicas will subscribe to this. Primary TP/RDB will send data to this manager service which can then take care of replicas. This way you can also keep all replica issues related handling logic at one place.

    Another benefit of this approach is this does not require to change primary TP/RDB services/logic to handle data issues. Manager service will handle everything.

Upvotes: 1

terrylynch
terrylynch

Reputation: 13572

The simplest approach would be to have the tickerplant log location and HDB location cross-mounted and accessible from both hosts. Then the RDB instance on the second host just has to replay the tickerplant log as usual, and subscribe to the same tickerplant - but the important thing is to not have the second RDB do an end-of-day writedown. Its .u.end should just clear the data from memory.

The second HDB wouldn't have any special conditions that I can think of, however in order for the instance to automatically refresh/reload (and pick up the newest date slice) then the original RDB which is doing the write-down would also need to trigger the refresh/reload of this second HDB.

Upvotes: 1

Related Questions