CODEWITHSUNDEEP
apache-kafkaapache-kafka-streams
Raman
Raman

Reputation: 717

Kafka Streams: Is GlobalStore threadsafe?

Can two parallel processors in a topology perform put/get on the same globalstore? For example, in a sample topology like below, one processor is traversing through the store using (KeyValueIterator) while the other processor is making a put/delete?

Topology:

Source: source (topic: [input topic with 6 partitions])
    --> processor1
    --> processor2

Processor: processor1
    <-- source

Processor: processor2
    <-- source      

Global Stores:
  GlobalStore: globalstoresource (topic: [input topic with 1 partition])
      --> loadglobalstore (store: myglobalstore)

Upvotes: 1

Views: 610

Answers (1)

Matthias J. Sax
Matthias J. Sax

Reputation: 62285

Global store are thread safe. However, only the "global processor" is supposed to do write the store. All other threads, should only read the store.

Note, that global stores are back the "global store input topic". If you do puts from a different thread, those put will only be done into the local store. Thus, if you run multiple instance, the other instanced will not see the change. Furthermore, if an error happens and the store needs to be recreated, the puts from the other thread will be lost because they are not backed by the "global store input topic".

Upvotes: 2

Related Questions