Elasticsearch "failed to find geo_point field [location]" when the mapping is there for that field

Question

I have an index with the following mapping:

{
  "mappings":{
    "my_stuff_type":{
      "properties":{
        "location": {
          "type": "geo_point",
          "null_value": -1
        }
      }
    }
  }
}

I have to use the property null_value because some of my documents don't have information about their location (latitude/longitude), but I still would like to search by distance on a location, cf. here: https://www.elastic.co/guide/en/elasticsearch/reference/current/null-value.html

When checking the index mapping details, I can verify that the geo mapping is there:

curl -XGET http://localhost:9200/my_stuff_index/_mapping | jq '.my_stuff_index.mappings.my_stuff_type.properties.location'
{
  "properties": {
    "lat": {
      "type": "float"
    },
    "lon": {
      "type": "float"
    }
  }
}

However when trying to search for documents on that index using a geo distance filter (cf. https://www.elastic.co/guide/en/elasticsearch/guide/current/geo-distance.html), then I see this:

curl -XPOST http://localhost:9200/my_stuff_index/_search -d'
{
  "query": {
    "bool": {
      "filter": {
        "geo_distance": {
          "location": {
            "lat": <PUT_LATITUDE_FLOAT_HERE>,
            "lon": <PUT_LONGITUDE_FLOAT_HERE>
          },
          "distance": "200m"
        }
      }
    }
  }
}' | jq

{
  "error": {
    "root_cause": [
      {
        "type": "query_shard_exception",
        "reason": "failed to find geo_point field [location]",
        "index_uuid": "mO94yEsHQseQDFPkHjM6tA",
        "index": "my_stuff_index"
      }
    ],
    "type": "search_phase_execution_exception",
    "reason": "all shards failed",
    "phase": "query",
    "grouped": true,
    "failed_shards": [
      {
        "shard": 0,
        "index": "my_stuff_index",
        "node": "MDueSn31TS2z0Lamo64zbw",
        "reason": {
          "type": "query_shard_exception",
          "reason": "failed to find geo_point field [location]",
          "index_uuid": "mO94yEsHQseQDFPkHjM6tA",
          "index": "my_stuff_index"
        }
      }
    ],
    "caused_by": {
      "type": "query_shard_exception",
      "reason": "failed to find geo_point field [location]",
      "index_uuid": "mO94yEsHQseQDFPkHjM6tA",
      "index": "my_stuff_index"
    }
  },
  "status": 400
}

I think the null_value property should allow me to insert documents without that location filed and at the same time I should be able to search with filters on that same "optional" field.

Why I am not able to filter on that "optional" field? How could I do this?

Edit:

To reproduce this issue with python run the following code snippet, before performing the curl/jq operations from the command line.

The python code depends on this: pip install elasticsearch==5.4.0.

from elasticsearch import Elasticsearch
from elasticsearch import helpers

my_docs = [
    {"xyz": "foo", "location": {"lat": 0.0, "lon": 0.0}},
    {"xyz": "bar", "location": {"lat": 50.0, "lon": 50.0}}
]

es = Elasticsearch([{'host': 'localhost', 'port': 9200}])

index_mapping = '''
{
  "mappings":{
    "my_stuff_type":{
      "properties":{
        "location": {
          "type": "geo_point",
          "null_value": -1.0
        }
      }
    }
  }
}'''

es.indices.create(index='my_stuff_index', ignore=400, body=index_mapping)

helpers.bulk(es, my_docs, index='my_stuff_index', doc_type='my_stuff_type')

Answer 1

as @Val has said you should change your mapping. If you define the location field in this way:

    "location": {
      "type": "geo_point"
    }

you could index lan and lon as two different subfield - without declaring them in the mapping as i shown - as described in the documentation - look here