CODEWITHSUNDEEP
elasticsearchnullkibana
Mahith Geallapally
Mahith Geallapally

Reputation: 21

ElasticSearch fetch NULL values as empty string or "N/A"

When i search for some document in elasticsearch, using GET /index/_search , I get the documents with some fields as null in them. Ex:

"_source" : {
    "ClientReference" : null,
    "SenderMSISDN" : null,
}

But I want in such a way so that null values are displayed as 'N/A' or empty string "" , similar to what IFNULL() does in an sql query. Is it possible? I've tried putting the mapping as the one below but this is not the solution

"ClientReference": {
          "type": "keyword",
          "null_value": "N/A"
        },

Upvotes: 0

Views: 2095

Answers (1)

Val
Val

Reputation: 217594

Elasticsearch will never modify the content of the source document you are indexing.

What "null_value": "N/A" does is that if ClientReference is null in your document, then ES will index N/A instead, but the source document will still contain "ClientReference": null.

The only way to modify a source document at indexing time is to leverage ingest pipelines

For instance, you can create such a pipeline that will set ClientReference to N/A if it's null

PUT _ingest/pipeline/unnullify
{
  "description" : "make sure to set default values to null fields",
  "processors" : [
    {
      "set" : {
        "if": "ctx.ClientReference == null",
        "field": "ClientReference",
        "value": "N/A"
      }
    }
  ]
}

Then when you index your document, you need to specify the pipeline like this:

PUT my-index/_doc/1?pipeline=unnullify
{
    "ClientReference" : null,
    "SenderMSISDN" : null,
}

Doing so would modify the source document and index it like this:

{
    "ClientReference" : "N/A",
    "SenderMSISDN" : null,
}

If you're using Logstash to index your data you need to configure your elasticsearch output plugin to use the ingest pipeline, like this:

output {
    elasticsearch {
        ...
        pipeline => 'unnullify'
        ...
    }
}

Upvotes: 0

Related Questions