elasticsearch date type mapping conflict

Question

I have an index with a field I am storing date information in. The field is in conflict at the moment. As far as I can tell there are three kinds of values:

• some documents don't have the field
• some documents have the field like this in the JSON:
  • "timestamp": "2019-03-01T23:32:28Z"
• other documents have the field like this in the JSON:
  • "timestamp": "1551206688760"

I want to fix the conflict.

I've tried doing a wildcard search and I get the following error:

failed to parse date field [*] with format [strict_date_optional_time||epoch_millis]

I have two questions, ultimately.

1) Is the core problem causing the conflict that when I tried to represent the timestamp in epoch_millis that I used a string rather than a number? IOW, "timestamp": 1551206688760 would have been good?

2) What is the proper way to fix this without simply tossing all the data out?

Answer 1

Unfortunately you will need to reindex.

1. Create new index with date mapping to provide multiple formats

"format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"

2. Reindex your data
3. Use aliases