CODEWITHSUNDEEP
elasticsearchnestedelasticsearch-plugin
joy d
joy d

Reputation: 418

Elastic search on nested field not working

We are using below settings with ES:

#Settings for elastic search
fos_elastica:
    clients:
        default: 
          host: %elastic_server_host%
          port: %elastic_server_port%
          headers: { Authorization: 'Basic %elastic_server_credential%' }
    indexes:
        organizations:
            settings:
                index:
                    analysis:
                        analyzer:
                            classic_analyser:
                                type: custom
                                tokenizer: lowercase
                                filter   : [my_snow]
                            simple_analyser:
                                type: custom
                                tokenizer: lowercase
                            email_analyser:
                                type: custom
                                tokenizer: uax_url_email    
                        filter:
                            my_snow:
                                type : "snowball"
                                language : "English"
            client: default
            types:
                packages:
                    mappings:
                        id:
                          type: integer
                        trackingCode:
                            type: string
                            index: not_analyzed
                        damaged:
                          type: boolean
                        urgent:
                          type: boolean
                        perishable:
                          type: boolean
                        checkedOut:
                          type: date
                        created:
                          type: date
                        confirmationType:
                          type: integer
                        forwardedTrackingCode   :
                          type: string
                          index: not_analyzed
                        firstNote   : { type: string, analyzer: classic_analyser }
                        secondNote  : { type: string, analyzer: classic_analyser }
                        deleted:
                          type: date
                        recipientId:
                            type: nested
                            properties:
                                id:
                                  type: integer
                                internalId: { type: string, analyzer: simple_analyser }
                                firstName:  { type: string, analyzer: simple_analyser }
                                lastName:  { type: string, analyzer: simple_analyser }
                                email: { type: string, analyzer: email_analyser }
                                mailbox:  { type: string, analyzer: simple_analyser }
                                phoneNumber:
                                  type: string
                                  index: not_analyzed
                                mobileNumber:
                                  type: string
                                  index: not_analyzed
                                jointAccountHolder:  { type: string, analyzer: simple_analyser }
                                deliveryMode: ~
                                deleted:
                                   type: date
                                address:
                                   type: nested
                                   properties:
                                     address:
                                       type: nested
                                       properties:
                                         country:
                                           type: string
                                           index: not_analyzed
                                         organisationName:
                                           type: string
                                           index: not_analyzed
                                         administrativeArea:
                                           type: string
                                           index: not_analyzed
                                         locality:
                                           type: string
                                           index: not_analyzed
                                         postalCode:
                                           type: string
                                           index: not_analyzed
                                         thoroughfare:
                                           type: string
                                           index: not_analyzed
                                         premise:
                                           type: string
                                           index: not_analyzed
                        carrierId:
                            type: nested
                            properties:
                              id:
                                type: integer 
                              slug:
                                type: string
                                index: not_analyzed
                              name:
                                type: string
                                index: not_analyzed
                              phone:
                                type: string
                                index: not_analyzed
                              otherName:
                                type: string
                                index: not_analyzed
                              webUrl:
                                type: string
                                index: not_analyzed
                    persistence:
                        driver: orm
                        model: MyCode\PackageBundle\Entity\Packages
                        finder: ~
                        provider: ~
                        listener: ~
                        repository: MyCode\PackageBundle\Entity\ElasticaPackagesRepository

                recipients:
                    mappings:
                        id:
                          type: integer
                        internalId: { type: string, analyzer: simple_analyser }
                        firstName:  { type: string, analyzer: simple_analyser }
                        lastName:  { type: string, analyzer: simple_analyser }
                        email: { type: string, analyzer: email_analyser }
                        mailbox:  { type: string, analyzer: simple_analyser }
                        phoneNumber:
                          type: string
                          index: not_analyzed
                        mobileNumber:
                          type: string
                          index: not_analyzed
                        jointAccountHolder:  { type: string, analyzer: simple_analyser }
                        deliveryMode: ~
                        deleted:
                          type: date
                        address:
                           type: nested
                           properties:
                             address:
                               type: nested
                               properties:
                                 country:
                                   type: string
                                   index: not_analyzed
                                 organisationName:
                                   type: string
                                   index: not_analyzed
                                 administrativeArea:
                                   type: string
                                   index: not_analyzed
                                 locality:
                                   type: string
                                   index: not_analyzed
                                 postalCode:
                                   type: string
                                   index: not_analyzed
                                 thoroughfare:
                                   type: string
                                   index: not_analyzed
                                 premise:
                                   type: string
                                   index: not_analyzed
                    persistence:
                        driver: orm
                        model: MyCode\RecipientBundle\Entity\Recipient
                        finder: ~
                        provider: ~
                        listener: ~
                        repository: MyCode\RecipientBundle\Entity\ElasticaRecipientRepository

In short, we are using two different types --- packages & recipients. In addition, recipient associated with a package is also stored in ES.

Now, if we search like :

{
  "query": {
    "query_string": {
      "query": "[email protected]",
      "fields": [
        "packages.recipientId.email"
      ],
      "default_operator": "OR"
    }
  },
  "size": "50",
  "from": "0",
  "sort": {
    "id": {
      "order": "asc"
    }
  }
}

ES is not returning any record although that particular recipient do exists. But if we search like :

{
  "query": {
    "query_string": {
      "query": "[email protected]",
      "fields": [
        "recipients.email"
      ],
      "default_operator": "OR"
    }
  },
  "size": "50",
  "from": "0",
  "sort": {
    "id": {
      "order": "asc"
    }
  }
}

ES is returning the record. I am not getting why search is behaving differently in above two cases.

Can you please suggest what we need change/correct so that search works in both the cases ?

Upvotes: 0

Views: 715

Answers (1)

lbrazier
lbrazier

Reputation: 2654

You have to do a nested query to query nested fields:

"Because nested docs are always masked to the parent doc, the nested docs can never be accessed outside the scope of the nested query." http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-nested-type.html http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-nested-query.html

Upvotes: 2

Related Questions