CODEWITHSUNDEEP
streamwso2complex-event-processingsiddhi
samser
samser

Reputation: 83

Siddhi JOIN events ALSO after window is EXPIRED

According to an article from WSO2 website when you JOIN events from two streams then following happens:

when an event from one stream reaches the In-Stream Join Processor, it is matched against all the available events of the other stream's Window Processor. When a match is found, those matched events are then sent to the Query Projector to create the output in-events; at the same time, the original event will be added to the Window Processor and it will remain there until it expires. Similarly, when an event expires from its Window Processor, it is matched against all the available events of the other stream's Window Processor; when a match is found, those matched events are sent to the Query Projector to create the output expired-events.

Basically, an event will be matched against other stream's window's events when it arrives and again will be looking for new matches when it's expired from its window. But this is not the behavior I'm noticing in a testing environment that I've setup. Here's my query:

FROM first_names#window.time(1 min) AS fst
UNIDIRECTIONAL JOIN last_names#window.time(2 min) AS lst
ON fst.identifier == lst.identifier
SELECT
  fst.identifier,
  fst.firstName,
  lst.lastName
INSERT INTO full_names

Then I publish the following events in the order laid out below to their respective streams:

{
  "lastName": "Colbert",
  "identifier": 1
}

{
  "firstName": "Stephen",
  "identifier": 1
}

{
  "lastName": "Carell",
  "identifier": 1
}

When the second event arrives, its counterpart already exists in the other stream's window so they match and the joined event is emitted right away as expected:

{
  "firstName": "Stephen",
  "lastName": "Colbert",
  "identifier": 1
}

Then the newly arrived event also gets stored in its stream window for one minute:

{
  "firstName": "Stephen",
  "identifier": 1
}

When the one minute is up and this event is being expired, a new counterpart for it exists in the other stream's window:

{
  "lastName": "Carell",
  "identifier": 1
}

So based on that article it should be matched with it and a new joined event sent as well looking like:

{
  "firstName": "Stephen",
  "lastName": "Carell",
  "identifier": 1
}

BUT, this event never arrives and the flow does not behave as explained in that article!!!

Any idea what might be causing this or if the article's claim is accurate and representative of the WSO2 Siddhi behavior? Because I did not see this in the official docs or other articles, so I'm a bit suspicious of this. Thanks in advance, I highly appreciate your help.

Upvotes: 0

Views: 402

Answers (1)

Grainier
Grainier

Reputation: 1654

Behaviour explained in the above article is accurate. The issue seems to be with your query. In your test query, you are using INSERT INTO full_names, so only current events will get emitted to the stream. However, events that get joined with an expired event will get emitted to the stream as expired events. So if you want to get those expired events, you have to mention it specifically using its output event category (i.e INSERT EXPIRED EVENTS INTO full_names or INSERT ALL EVENTS INTO full_names). Please refer to the following documentation to get to know more about output event categories.

Upvotes: 1

Related Questions