CODEWITHSUNDEEP
javaapache-storm
zenbeni
zenbeni

Reputation: 7213

When using field grouping in storm, is there a limit to known field values or a timeout?

I use a field grouping with Apache Storm, it works well.

But if I have an infinite number of field values, it means that storm has to keep track of an infinite number of values in its bolts! Otherwise, some values will go to the wrong bolt and it will destroy any caching technique.

I suppose there is a cache somewhere in the bolt with an expiration system or/and a limitation on number of field values to check for bolt field grouping. Is it possible to tune it / overwrite it?

Upvotes: 4

Views: 659

Answers (1)

gasparms
gasparms

Reputation: 3354

You don't need cache nothing, fieldsGrouping uses a mod hash function to determine which task to send a tuple, so you can be sure that will be processed in the correct task.

Upvotes: 4

Related Questions