CODEWITHSUNDEEP
javaapache-storm
breezymri
breezymri

Reputation: 4343

Apache storm reliability: would ack/fail a tuple with changed values guarantee reliability?

I'm working on a project that inherits from a custom-made legacy storm bolt. The bolt is supposed to be reliable and it acks or fails the tuple based on the success or failure of some operations. The problem is that in the transformation, the tuple values get changed. Sample code:

public void execute(Tuple tuple) {
  object newValues = transformTuple(tuple);
  tuple.getValues().set(0, newValues);
  try {
    // some other operation
    ...
    collector.ack(tuple);
  } catch (Exception e) {
    collector.fail(tuple);
  }
}

This is suspicious since it's acking/failing a tuple with changed values. I couldn't find any documentation on whether only the key of the tuple is used in acking or both key and values. So my question is: would such ack/fail work to guarantee reliability (retry if fail)?

Upvotes: 2

Views: 284

Answers (1)

Stig Rohde Døssing
Stig Rohde Døssing

Reputation: 3651

It doesn't matter. Storm doesn't track the tuple contents. If you want to know how Storm tracks tuples, take a look at https://storm.apache.org/releases/2.0.0-SNAPSHOT/Guaranteeing-message-processing.html, in particular the section "How does Storm implement reliability in an efficient way?".

The tl;dr for that section is that Storm keeps track of one 64-bit number (the "ack val") for the entire tuple tree. When you emit a new tuple (i.e. a new edge in the tree), Storm generates a random id for that edge, and XORs it onto the ack val. When the edge gets acked by the receiving bolt, the same ID is XORd onto the ack val again. Since a XOR b XOR b is a, Storm ends up with the ack val being 0 when all the edges are processed.

Since acking only depends on the tuple ids (which you have no reason to want to change), it doesn't matter what you do with the tuple before acking it.

Upvotes: 2

Related Questions