Querying nested fields with analyzer results in error

Question

I tried to use a synonym analyzer for my already working elastic search type. Here's the mapping of my serviceEntity:

{
"serviceentity" : {
    "properties":{          
        "ServiceLangProps" : {
            "type" : "nested",
            "properties" : {
                "NAME"          : {"type" : "string", "search_analyzer": "synonym"},
                "LONG_TEXT"     : {"type" : "string", "search_analyzer": "synonym"},
                "DESCRIPTION"   : {"type" : "string", "search_analyzer": "synonym"},
                "MATERIAL"      : {"type" : "string", "search_analyzer": "synonym"},
                "LANGUAGE_ID"   : {"type" : "string", "include_in_all": false}
            }
        },
        "LinkProps" : {
            "type" : "nested",
            "properties" : {
                "TITLE" : {"type" : "string", "search_analyzer": "synonym"},
                "LINK"  : {"type" : "string"},
                "LANGUAGE_ID"   : {"type" : "string", "include_in_all": false}
            }
        },
        "MediaProps" : {
            "type" : "nested",
            "properties" : {
                "TITLE"     : {"type" : "string", "search_analyzer": "synonym"},
                "FILENAME"  : {"type" : "string"},
                "LANGUAGE_ID"   : {"type" : "string", "include_in_all": false}
            }
        }
    }
}

}

And these are my setting

 {
  "analysis": {
    "filter": {
      "synonym": {
        "ignore_case": "true",
        "type": "synonym",
        "synonyms": [
          "lorep, spaceship",
          "ipsum, planet"
        ]
      }
    },
    "analyzer": {
      "synonym": {
        "filter": [
          "lowercase",
          "synonym"
        ],
        "tokenizer": "whitespace"
      }
    }
  }
}

When In try to search for anything, I get this Error:

Caused by: org.elasticsearch.index.query.QueryParsingException: [nested] nested object under path [ServiceLangProps] is not of nested type

And I don't understand why. If I don't add any analyzer to my setting, everything works fine.

I'm using the java API to communicate with the elasticsearch instance. Therefore my code looks something like this for the multi match query:

MultiMatchQueryBuilder multiMatchBuilder = QueryBuilders.multiMatchQuery(fulltextSearchString, QUERY_FIELDS).analyzer("synonym");

The query string created by the java API looks like this:

{
  "query" : {
    "bool" : {
      "must" : {
        "bool" : {
          "should" : [ {
            "nested" : {
              "query" : {
                "bool" : {
                  "must" : [ {
                    "match" : {
                      "ServiceLangProps.LANGUAGE_ID" : {
                        "query" : "DE",
                        "type" : "boolean"
                      }
                    }
                  }, {
                    "multi_match" : {
                      "query" : "lorem",
                      "fields" : [ "ServiceLangProps.NAME", "ServiceLangProps.DESCRIPTION", "ServiceLangProps.MATERIALKURZTEXT", "ServiceLangProps.DESCRIPTION_RICHTEXT" ],
                      "analyzer" : "synonym"
                    }
                  } ]
                }
              },
              "path" : "ServiceLangProps"
            }
          }, {
            "nested" : {
              "query" : {
                "bool" : {
                  "must" : [ {
                    "match" : {
                      "LinkProps.LANGUAGE_ID" : {
                        "query" : "DE",
                        "type" : "boolean"
                      }
                    }
                  }, {
                    "match" : {
                      "LinkProps.TITLE" : {
                        "query" : "lorem",
                        "type" : "boolean"
                      }
                    }
                  } ]
                }
              },
              "path" : "LinkProps"
            }
          }, {
            "nested" : {
              "query" : {
                "bool" : {
                  "must" : [ {
                    "match" : {
                      "MediaProps.LANGUAGE_ID" : {
                        "query" : "DE",
                        "type" : "boolean"
                      }
                    }
                  }, {
                    "match" : {
                      "MediaProps.TITLE" : {
                        "query" : "lorem",
                        "type" : "boolean"
                      }
                    }
                  } ]
                }
              },
              "path" : "MediaProps"
            }
          } ]
        }
      },
      "filter" : {
        "bool" : { }
      }
    }
  }
}

If I try it on the LinkProps or MediaProps, I get the same error for the respective nested object.

Edit: I'm using version 2.4.6 of elasticsearch

Answer 1

I just tried this

{
  "query": {
    "bool": {
      "must": {
        "query": {
          "multi_match": {
            "query": "foo",
            "fields": [
              "LinkProps.TITLE",
              "LinkProps.LINK"
            ],
            "analyzer": "synonym"
          }
        }
      }
    }
  }
}

As you can see, I removed the "nested" wrapper

"nested": {
          "path": "LinkProps",
          ...
}

which now leads at least in some results (Not sure yet, if these will finally be the correct results). I'm trying to apply this to the original project and keep you posted if this also worked.

Answer 2

I created a minimal example of what I'm trying to do.

My mapping looks like this:

{
    "serviceentity" : {
        "properties":{          
            "LinkProps" : {
                "type" : "nested",
                "properties" : {
                    "TITLE" : {"type" : "string", "search_analyzer": "synonym"},
                    "LINK"  : {"type" : "string"},
                    "LANGUAGE_ID"   : {"type" : "string", "include_in_all": false}
                }
            }
        }
    }
}

And my settings for the synonym analyzer in JAVA code:

XContentBuilder builder = jsonBuilder()
        .startObject()
            .startObject("analysis")
                .startObject("filter")
                    .startObject("synonym") // The name of the analyzer
                        .field("type", "synonym") // The type (derivate)
                        .field("ignore_case", "true") 
                        .array("synonyms", synonyms) // The synonym list
                    .endObject()
                .endObject()
                .startObject("analyzer")
                    .startObject("synonym")
                        .field("tokenizer", "whitespace")
                        .array("filter", "lowercase", "synonym")
                    .endObject()
                .endObject()
            .endObject()
        .endObject();

The metadata which the ElasticSearch Head Chrome plugin spits out looks like this:

{
  "analysis": {
    "filter": {
      "synonym": {
        "ignore_case": "true",
        "type": "synonym",
        "synonyms": [
          "Test, foo",
          "Title, bar"
        ]
      }
    },
    "analyzer": {
      "synonym": {
        "filter": [
          "lowercase",
          "synonym"
        ],
        "tokenizer": "whitespace"
      }
    }
  }
}

When I now use a search query to look for "Test" I get the same error as mentioned in my first post. Here's the query

{
  "query": {
    "bool": {
      "must": {
        "nested": {
          "path": "LinkProps",
          "query": {
            "multi_match": {
              "query": "Test",
              "fields": [
                "LinkProps.TITLE",
                "LinkProps.LINK"
              ],
              "analyzer": "synonym"
            }
          }
        }
      }
    }
  }
}

which leads to this error

{
  "error": {
    "root_cause": [
      {
        "type": "query_parsing_exception",
        "reason": "[nested] nested object under path [LinkProps] is not of nested type",
        "index": "minimal",
        "line": 1,
        "col": 44
      }
    ],
    "type": "search_phase_execution_exception",
    "reason": "all shards failed",
    "phase": "query",
    "grouped": true,
    "failed_shards": [
      {
        "shard": 0,
        "index": "minimal",
        "node": "6AhE4RCIQwywl49h0Q2-yw",
        "reason": {
          "type": "query_parsing_exception",
          "reason": "[nested] nested object under path [LinkProps] is not of nested type",
          "index": "minimal",
          "line": 1,
          "col": 44
        }
      }
    ]
  },
  "status": 400
}

When I check the analyzer with

GET http://localhost:9200/minimal/_analyze?text=foo&analyzer=synonym&pretty=true

I get the correct answer

{
  "tokens": [
    {
      "token": "foo",
      "start_offset": 0,
      "end_offset": 3,
      "type": "word",
      "position": 0
    },
    {
      "token": "test",
      "start_offset": 0,
      "end_offset": 3,
      "type": "SYNONYM",
      "position": 0
    }
  ]
}

So the analyzer seems to set up correctly. Did I messed up the mappings? I guess the problem is not because I have nested objects or is it?

Answer 3

Would be helpful to check the query string as well and knowing what version of ES is being used.

I couldnt see the synonyms_path as well as the fact you are using nested types can cause that error.

You probably have seen this already but in case you havent

https://www.elastic.co/guide/en/elasticsearch/reference/5.5/analysis-synonym-tokenfilter.html