CODEWITHSUNDEEP
phpelasticsearch
jones
jones

Reputation: 1453

Elasticsearch don't allow indexing null in date field

I have bellow data to index:

array:6 [▼
  "index" => "my_index"
  "type" => "audit_field"
  "id" => "57f36d28a1dfc"
  "parent" => "57f36d289f1b2"
  "routing" => 318
  "body" => array:19 [▼
     "session_id" => 318
     "trans_seq_no" => 13
     "table_seq_no" => 13
     "field_id" => 65
     "field_name" => "id"
     "new_value" => 45
     "old_value" => 45
     "date_type_new_value" => null
     "date_type_old_value" => null
     "time_type_new_value" => null
     "time_type_old_value" => null
 ]
]

I have defined mapping myself as bellow

array(
    'index' => 'promote_kmp',
    'type' => 'audit_field',
    'body' => [
        'audit_field' => [
            '_source' => [
                'type' => 'string'
            ],
            "_parent" => [
                "type" => "audit_table"
            ],
            'properties' => [
                'session_id'   => array('type' =>  'string'),
                'trans_seq_no' => array('type' =>  'string'),
                'table_seq_no' => array('type' =>  'string'),
                'field_id'     => array('type' =>  'string'),
                'field_name'   => array('type' => 'string'),
                'old_value'    => array(
                    'type' => 'string'
                ),
                'new_value'    => array(
                    'type' => 'string'
                ),
                'date_type_new_value' => array(
                    'type'   => 'date',
                    'format' =>'YYYY-MM-dd HH:mm:ss||YYYY-MM-dd||MM/dd/yyyy||yyyy/MM/dd'
                ),
                'date_type_old_value' => array(
                    'type'   => 'date',
                    'format' =>'YYYY-MM-dd HH:mm:ss||YYYY-MM-dd||MM/dd/yyyy||yyyy/MM/dd'
                ),
                'time_type_new_value' => array(
                    'type'   => 'date',
                    'format' => 'HH:mm:ss'
                ),
                'time_type_old_value' => array(
                    'type'   => 'date',
                    'format' => 'HH:mm:ss'
                )
            ]
        ]
    ]
);

But when i wan to index data with null in date field, elasticsearch shows and exception that can't parse date field value with mapping date format types. How to solve this, i have also tried with empty, but not working, and give the same message.

MapperParsingException[failed to parse [date_type_new_value]]; nested: 
MapperParsingException[failed to parse date field [], tried both date format 
[YYYY-MM-dd HH:mm:ss||YYYY-MM-dd||MM/dd/yyyy||yyyy/MM/dd], and timestamp 
number with locale []]; nested: IllegalArgumentException[Invalid format: ""];

Upvotes: 3

Views: 10185

Answers (2)

Jesal
Jesal

Reputation: 95

With ES 7, you have the option of specifying null_value as a field property. Null values cannot be searched, however. See the manual page on null-values for null_value configuration and the manual page on dates for the Date datatype.

Upvotes: 1

ChintanShah25
ChintanShah25

Reputation: 12672

I am not sure why null is not working, I can index null value in ES 1.7, also IllegalArgumentException[Invalid format: ""]; indicates blank string and not null value.

If you don't have control over the data, you need to use ignore_malformed which will not index the date field but index the document normally.

Upvotes: 1

Related Questions