CODEWITHSUNDEEP
elasticsearchlogstash
cybergoof
cybergoof

Reputation: 1437

ElasticSearch Index limitations

I am using Logstash to ingest custom logs. I want the index name of the document to be built off a field documentID, which is a GUID. I'm finding that ElasticSearch will store the document if "documentID" is all numbers or lowercase letters. If there is an uppercase, then it fails. Is there limitations to what kind of strings can be in an index?

input {
     tcp{
        port=>3362
        type="mf_data"
        codec=>"json_lines"
     }

}

 filter{
    json{source=>"message"}
    grok{match=>"message","documentID:%{DATA:documentID}"]}
 }
  output{
    elasticsearch{
       host=>"localhost"
       index_type=>"customType"
       index=>"event_%{documentID}"
    }
  }

Input is {"domain":"test.com","documentID":"cAmii"}

Upvotes: 0

Views: 364

Answers (1)

Alain Collins
Alain Collins

Reputation: 16362

Check this post:

Index names are limited by the file system. They may only be lower case, and my not start with an underscore. While we don't prevent index names starting with a ., we reserve those for internal use. Clearly, . and .. cannot be used.

These characters are already illegal: \, /, *, ?, ", <, >, |, , ,. We should also add the null byte.

There are other filenames which are illegal in Windows, but we probably don't need to check for those.

Upvotes: 1

Related Questions