CODEWITHSUNDEEP
logstash
Lovika
Lovika

Reputation: 617

How to load CSV file in logstash

I am trying to load CSV file in logstash but it is not reading the file and not creating the index in elasticsearch I need to read the CSV file in elasticsearch. Tried few changes in config file.

My Config file

input {
    file {
        type => "csv"
        path => "/root/installables/*.csv"
        start_position => beginning
    }
}

filter {
    grok {
        match => { "message" => "%{COMBINEDAPACHELOG}" }
    }
}

output {
    elasticsearch {
        hosts => localhost
        index => "client"
    }
}

Could anybody tell how to load CSV file in logstash?

Upvotes: 5

Views: 14199

Answers (1)

rhernando
rhernando

Reputation: 1071

I think you should put a "csv" filter. I make it work like this:

input {
  file {
    path => "/filepath..."
    start_position => beginning
    # to read from the beginning of file
    sincedb_path => "/dev/null"
  }
}

filter {
    csv {
        columns => ["COL1", "COL2"]
    }
}

output {
    stdout { codec => rubydebug }
    elasticsearch {
        host => "localhost"
        index => "csv_index"
    }
}

Also, adding stdout as output helps you to debug and know if the file is loading

Upvotes: 10

Related Questions