Index only a few fields in a ElasticSearch mapping

Question

I'm working with ElasticSearch 6.5. I have indexed a csv file using the following code:

def create_index(es_object, index_name):
    created = False
    # index settings
    settings = {
        "settings": {
            "number_of_shards": 1,
            "number_of_replicas": 0,
            "analysis": {
                "filter": {
                    "dbl_metaphone": { 
                        "type":    "phonetic",
                        "encoder": "beider_morse"
                    }
                },
                "analyzer": {
                    "dbl_metaphone": {
                        "tokenizer": "standard",
                        "filter":    "beider_morse"
                    }
                }
            }
        },
        "mappings": {
            "test": {
                #"dynamic": "strict",
                "properties": {
                    "family name": {
                        "type": "text",
                        "index": "analyzed",
                        "fields": {
                            "phonetic": { 
                                "type":     "string",
                                "analyzer": "dbl_metaphone"
                            }
                        }
                    },
                    "Firstname": {
                        "type": "text",
                        "index": "analyzed",
                        "fields": {
                            "phonetic": { 
                                "type":     "string",
                                "analyzer": "dbl_metaphone"
                            }
                        }
                    },

                "Date of birth": {
                    "type": "text",
                    "index": "false"
                },  
                "Place of birth": {
                    "type": "text",
                    "index": "false",
                },

            }
        }
    }
}

    try:
        if not es_object.indices.exists(index_name):
            # Ignore 400 means to ignore "Index Already Exist" error.
            es_object.indices.create(index=index_name, ignore=400, body=settings)
            print('Created Index')
        created = True
    except Exception as ex:
        print(str(ex))
    finally:
        return created

The problem is that when I tried to search the data with kibana, all the field are searchable and aggregatable. And i wanted to exclude "Date of birth" and "Place of birth" from searchable and aggregatable.

Can anyone explain whats the problem with my mapping and how to update the index to achieve it?

Thank you

Answer 1

You need to assign the index as "not_analyzed"

"Date of birth": { "type": "text", "index" : "not_analyzed" },

Answer 2

Let's try this with a minimal example (added through Console in Kibana, but you'll be able to change that to plain curl commands quite easily):

PUT test
{
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 0
  },
  "mappings": {
    "_doc": {
      "dynamic": "strict",
      "properties": {
        "family name": {
          "type": "text",
          "index": "true"
        },
        "Firstname": {
          "type": "text",
          "index": "true"
        },
        "Date of birth": {
          "type": "text",
          "index": "false"
        }
      }
    }
  }
}

PUT /test/_doc/1
{
  "family name": "foo",
  "Firstname": "bar",
  "Date of birth": "baz"
}

This works for me. I can find foo and bar, but not baz:

Also after refreshing the index pattern the date of birth field is neither searchable nor aggregateable:

Some quick observations:

1. index is true or false. Its on a different field, so probably not related to your issue.
2. Unless you want to suffer in the future I would avoid spaces in field names. I can just see to many things going wrong.