CODEWITHSUNDEEP
elasticsearchlogstashelk
sunjiayang
sunjiayang

Reputation: 3

logstash is started but i can't get the result in the output

I've written a conf file for logstash as shown below:

input {
    file {
        mode => "tail"
        path => ["/Users/sunjiayang/Documents/workspace_logstash/logstash-7.1.0*_log.csv"]
        sincedb_path => "/dev/null"
        start_position => "beginning"
        codec => plain { 
            charset => "UTF-8"
        }
    }
}

filter {
    csv {
        columns => ["Date", "Level", "ErrorMessage","UserId"]
        convert => {
            "UserId" => "integer"
        }
        skip_header => true
    }
    date {
        match => ["Date", "yyyy-MM-dd HH:mm:ss"]
    }
}

output {
    # elasticsearch {
    #     hosts => ["localhost:9200"]
    #     index => "log"
    # }
    stdout {
        codec => rubydebug
    }
}

I ran the following command:

command

sudo bin/logstash -f config/logstash.conf

but I've only got this success info:

output

Sending Logstash logs to /Users/sunjiayang/Documents/workspace_logstash/logstash-7.1.0/logs which is now configured via log4j2.properties
[2021-12-03T23:15:51,175][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2021-12-03T23:15:51,201][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.1.0"}
[2021-12-03T23:16:00,201][INFO ][logstash.javapipeline    ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>500, :thread=>"#<Thread:0x42001e06 run>"}
[2021-12-03T23:16:00,773][INFO ][logstash.javapipeline    ] Pipeline started {"pipeline.id"=>"main"}
[2021-12-03T23:16:00,903][INFO ][filewatch.observingtail  ] START, creating Discoverer, Watch with file and sincedb collections
[2021-12-03T23:16:00,906][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2021-12-03T23:16:01,497][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}

I want the result like this, what should I do?

{
            "Date" => "2019-05-24 10:00:00",
        "@version" => "1",
            "path" => "/Users/myuser/work/elasticsearch/log/20190525_log.csv",
    "ErrorMessage" => "Success.",
            "host" => "local",
      "@timestamp" => 2019-05-25T04:00:00.000Z,
           "Level" => "INFO",
          "UserId" => 1,
         "message" => "2019-05-24 10:00:00,INFO,Success.,1"
}

Upvotes: 0

Views: 1167

Answers (1)

Musab Dogan
Musab Dogan

Reputation: 3680

I think the problem because of the input. Note: Logstash works in linux "tail -f" logic Try this

input { stdin {} }
filter {}
output { stdout {} }

and write something to the terminal, if you can see the result you are in the right way.

Change conf file and add input { file { ... }} output { stdout {} } without filter and try again to see the result.

if you can't see the result check your input {} again, else check filter {}

config-examples

Upvotes: 1

Related Questions