CODEWITHSUNDEEP
elasticsearchfiltertimestampaggregation
Eli
Eli

Reputation: 4926

Aggregate over the maximum timestamp lower than value

So, we trying to aggregate all documents by color, where for each color, find the maximum timestamp as max_timestamp. Then, filter only buckets where max_timestamp is lower then now-5m. The idea here is to check if there is any color without reported document from the last 5 minutes.

Here is what we got by now:

{
  "size": 0,
    "aggs": {
      "colors_aggs": {
        "terms": {
          "field": "color",
            "size": 10
          },
          "aggs": {
            "max_timestamp": {
              "max": {
                "field": "timestamp"
              }
            },
            "aggs": {
              "filter": {
                "range": {
                  "timestamp": {
                    "lt": "now-5m"
                  }
                }
              }
            }
          }
        }
      }
    }

It seems to ignore the third aggregation. Buckets with timestamp greater than now-5m are shown.

Any help?

Upvotes: 5

Views: 1437

Answers (2)

hanetz
hanetz

Reputation: 323

Perhaps you can use script to filter-out the unwanted records (in your case records with timestamp > "now - 5m") within the final aggregation, and then your final aggregation (and eventually the output) will be based only on the wanted records. The query should be something like this:

{
  "size": 0,
  "aggs": {
    "colors_aggs": {
      "terms": {
        "field": "color",
        "size": 10
      },
      "aggs": {
        "maximals": {
          "max": {
            "field": "timestamp":
          }
        },
        "max_bucket_filter": {
          "bucket_selector": {
            "buckets_path": {
              "maxs": "max_timestamp"
            },
            "script": {
              "lang": "expression",
              "script": "maxs < [CurrentUnixTime x 1000]"
            }
          }
        }
      }
    }
  }
}

Notice that the above script cannot accept the keyword now-5m so you will have to set the current unix time every time you execute the query.

Upvotes: 2

Val
Val

Reputation: 217304

You need to include the max_timestamp aggregation as a sub-aggregation of the filter one. Try like this:

{
  "size": 0,
  "aggs": {
    "colors_aggs": {
      "terms": {
        "field": "color",
        "size": 10
      },
      "aggs": {
        "5m": {
          "filter": {
            "range": {
              "timestamp": {
                "lt": "now-5m"
              }
            }
          },
          "aggs": {
            "max_timestamp": {
              "max": {
                "field": "timestamp"
              }
            }
          }
        }
      }
    }
  }
}

UPDATE

If you want the now-5m filter to apply to the whole aggregation section then you can move it as the top aggregation, like this:

{
  "size": 0,
  "aggs": {
    "5m": {
      "filter": {
        "range": {
          "timestamp": {
            "lt": "now-5m"
          }
        }
      },
      "aggs": {
        "colors_aggs": {
          "terms": {
            "field": "color",
            "size": 10
          },
          "aggs": {
            "max_timestamp": {
              "max": {
                "field": "timestamp"
              }
            }
          }
        }
      }
    }
  }
}

Upvotes: 0

Related Questions