Analyze, anaomaly detection and alert after logs analysis - ELK

Question

We have logs from multiple applications and vm's streamed to ELK(elastic search, kibana, logstash) stack. This is in our data center, not on the cloud. I am looking for advice on a) Any mechanism or service that run on top of this ELK stack and understand the application behavior b) If there is an anomaly, generate the alerts

This system/service needs to understand the application (machine learning capabilities). When I say understand the application, I would like the service to understand my application has high traffic on Wednesday and low traffic on Friday. If on Friday, I am seeing a very high traffic then that is an anomaly. Another example: If my application has been throwing 20 exceptions a day and now I am aseeing 50 exceptions after the latest version was deployed, this is an anomaly.

b) If there is an anomaly, send the alert via pagerduty or email to all the stakeholders.

This could be a paid service or something that can be plugged in with ELK installation. I do not want to ships the logs to cloud to get this service.

Answer 1

Machine learning, anomaly detection, and alerting are all features of Elastic's x-pack.