CODEWITHSUNDEEP
wso2complex-event-processing
user2656744
user2656744

Reputation: 21

Complex Event Processor 2.1.0 input streams

I'm new at the CEP 2.1 and my question is related to time-frame that tha CEP hold on to the input stream

let say that you regularly send data to some input stream let's say "HELLOSTREAM". for how long does the CEP save the inputs to the stream what is the max time etc...

let say if I send data every day for 365 days will I get back all data on the 366 day or will he truncate the data at some point( will hold ony last 100 days) ? no matter what time-window I set in the query ?

is there a limit ?

Upvotes: 2

Views: 198

Answers (2)

Rajeev Sampath
Rajeev Sampath

Reputation: 2747

To add to @Mohanadarshan's answer, if what you want is to extract and store some values from a fast event stream over a long period, a better CEP-based approach will be to use persistent event tables (which will be included in the upcoming CEP version 3.0.0 which will be released soon). This way you'll be able to do real-time processing against some extracted and persisted data. But as @Mohanadarshan has mentioned, if your requirement is batch processing (and if you do not need to detect anything real-time), WSO2 BAM will be a better option.

Using sliding windows over a very long period to store large amounts of data is not a good idea as they are stored in memory and also you'll loose data if server goes down.

Upvotes: 1

Mohanadarshan
Mohanadarshan

Reputation: 830

CEP is a real-time processing server. It is used to find pre-defined pattern in real-time and for realtime monitoring. It keeps the data in memory and process the events but you can persist the data to cassandra for distributed processing...

Here data will be keep in the memory based on the window size that you defined, It depends on the window type that you are using and time or length given to that window... If you are not using any window it will not keep any data in memory...

If you want to store data for 365 days or 100 days, then it is not a real-time use-case. For that you have to use offline processing server like BAM.

Upvotes: 1

Related Questions