CODEWITHSUNDEEP
sortingelasticsearch
Scarabas
Scarabas

Reputation: 103

ElasticSearch 5: sorting by nested key and value results in "Fielddata is disabled on text fields by default"

I'm trying to sort a search result in ElasticSearch 5 by a nested set of key value pairs. Ie.:

I have the following result (pseudo structure, for the sake of keeping it simple):

{
  "hit1": {
    "nested_objects": [
      {
        "Key": "abc",
        "Value": 0.1
      },
      {
        "Key": "def",
        "Value": 0.3
      }
    ]
  },
  "hit2": {
    "nested_objects": [
      {
        "Key": "abc",
        "Value": 0.9
      },
      {
        "Key": "def",
        "Value": 0.1
      }
    ]
  }
}

I'd like that to be sorted by "highest Value where Key = 'abc'", which would mean that "hit2" would come out on top.

My mapping is as follows:

{
  "test_index": {
    "mappings": {
      "test_type": {
        "properties": {
          "nested_objects": {
            "type": "nested",
            "properties": {
              "Key": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "Value": {
                "type": "float"
              }
            }
          }
        }
      }
    }
  }
}

I've tried to follow the following suggestions:

Sort elastic search query based on a nested key value array

Sort nested object in Elasticsearch

...but I consistently get the following error:

"Fielddata is disabled on text fields by default"

An example of a sorting attempt that reproduces this:

{
  "sort": [
    {
      "nested_objects.Key": {
        "order": "desc",
        "nested_path": "nested_objects",
        "nested_filter": {
          "term": { "nested_objects.Key": "abc" }
        }
      }
    }
  ]
}

What is the best practice way of solving this? Is enabling field data (which uses a lot of ram) really the only option here?

Upvotes: 1

Views: 728

Answers (2)

Taras Kohut
Taras Kohut

Reputation: 2555

The Text datatype is analyzed (this type is used for full text search). Sorting on analyzed fields is expensive, that's why it is disabled by default.
Usually you don't need to sort by analyzed field, so try sorting on keyword field instead: 

"term": { "nested_objects.Key.keyword": "abc" }

Read this doc to understand the problem more deeply.

Upvotes: 1

The_Torst
The_Torst

Reputation: 401

In your example, you are sorting by the Key, but you say you want to sort by Value. Key is not sortable by default (as Taras points out, you'd have to use ".keyword" to sort on that field), which throws the error you encounter.

To solve your problem completely, sort by Value and filter by Key. You're already filtering by Key (you don't need to use ".keyword" to do so), so all you have to do is to sort by Value:

{
  "sort": [
    {
      "nested_objects.Value": {   <-- SOLUTION
        "order": "desc",
        "nested_path": "nested_objects",
        "nested_filter": {
          "term": { "nested_objects.Key": "abc" }
        }
      }
    }
  ]
}

Upvotes: 1

Related Questions