CODEWITHSUNDEEP
elasticsearchautocomplete
Jotschi
Jotschi

Reputation: 3632

How to handle auto completion on multi word text?

My input text is a multiword english text and I have the requirement to implement a autocompletion feature for that text.

I initially looked at search completion suggesters only to figure out that those can only match the first characters of the input. This is fine for auto completion of product names or address but not very useful when requiring a auto completion on any word in the input text.

After that I setup an edge_ngram analyzer and query to locate those documents which contain the input string. That works just fine but I don't know how to use this information to provide options for my auto completion.

I could use a highlighter in order to show the words which match the query. That data could in turn be used to setup a list of options. This solution seems rather hacky and not very elegant and I wonder how this problem is usually solved?

I'm unfortunately not able to maintain another field which could include the auto completion options for the documents.

Upvotes: 2

Views: 1157

Answers (2)

Jotschi
Jotschi

Reputation: 3632

I'm currently using highlight information of the query in order to construct the autocomplete options.

My Query:

{
  "query": {
    "match": {
      "fields.content.auto": {
        "query": "content co",
        "analyzer": "standard"
      }
    }
  },
  "highlight": {
    "fields": {
      "fields.content.auto": {
        "fragment_size": 0,
        "number_of_fragments": 10,
        "pre_tags" : [ "%ha%" ],
        "post_tags" : [ "%he%" ]
      }
    }
  },
  "_source": ["uuid", "language"]
}

My auto field used the autocomplete analyzer.

"auto": {
  "type": "string",   
  "analyzer": "autocomplete"
}

And this is the index configuration that I'm using:

{
  "analysis": {
    "filter": {
      "my_stop": {
        "type":       "stop",
        "stopwords":  "_english_"
      },
      "autocomplete_filter": {
          "type": "edge_ngram",
          "min_gram": 1,
          "max_gram": 20
      }
    },
    "analyzer": {
      "autocomplete": {
        "type": "custom",
        "tokenizer": "standard",
        "filter": [
          "lowercase",
          "my_stop",
          "autocomplete_filter"
        ]
      }
    }
  }
}

The solution was mainly inspired by the Search-as-you-type post.

I process the response JSON in order to get the autocomplete options. The highlight information is used to extract all found tokens. These tokens are next used to construct the potential autocomplete phrase by also comparing it to the phrase that the user has already entered. The neat thing is that a stop word filter can be applied and thus stopwords will never be highlighted and in turn never be used for autocomplete suggestions.

A PoC Java code of this processor can be found here

I'm not yet sure whether I'll run with this solution but I want to share it anyway.

Upvotes: 2

LaserJesus
LaserJesus

Reputation: 8560

I think your best option is to create a dedicated index for storing just the suggestions using the edge_ngram analyzer. If you use the completion suggesters you need to explicitly define your actual suggestions anyway. The completion suggester is also document centric in ES 5.x so if you index multiple documents with the same suggestions you will get duplicate suggestions returned on a match. There is a de-duplication option in ES 6, but that has only just been released.

If you have a dedicated suggestion index you can use a hash of the suggestion as a document ID to avoid duplicates. You can start indexing document titles and other useful meta data as suggestions. Later on you could include historical searches entered by users that are seen as successful due to the user ultimately clicking on or purchasing the returned results.

Upvotes: 0

Related Questions