CODEWITHSUNDEEP
elasticsearchlogstash
Joseph
Joseph

Reputation: 2245

Use Logstash to parse million logs

I want to be able to parse a million JSON files (static and not being written to by any process) and send to elasticsearch using ELK. However I run into an error:

failed to open /opt/data/input/[email protected]: Permission denied -

It seems logstash is tailing all the files it's opened and therefore can't open anymore. I want to be able to parse these files and have logstash forget them. No need to "tail" them. Is this possible? I see a JIRA opened for this at https://logstash.jira.com/browse/LOGSTASH-1929. Does anyone else face this problem? Has this been solved or is it an open issue?

Upvotes: 0

Views: 185

Answers (1)

James Boutcher
James Boutcher

Reputation: 2613

The error you gave makes it look like the user logstash is running as doesn't have permissions to read the files matching the pattern you are looking for. Or, maybe they don't have access while the file is being written (for a brief moment in time).

Upvotes: 1

Related Questions