CODEWITHSUNDEEP
dockerloggingkuberneteselastic-stackfluentd
Ganang Wahyu W
Ganang Wahyu W

Reputation: 59

Fluentd in docker cannot tail from my log file

Fluentd in Docker cannot tail from my log file

My input in /var/log/logf/a.log:

test 1st log
test 2nd log
test 3rd log

and my config in /opt/app/conf/fluent.conf:

<source>
   @type tail
   path /var/log/logf/a.log
   tag test
   read_from_head true
   <parse>
      @type none
      message_key test
   </parse>
</source>

<match test>
   @type stdout
</match>

and my Dockerfile id /opt/app/Dockerfile

FROM fluent/fluentd:v1.11-debian-1
USER root

COPY ./conf/fluent.conf /fluentd/etc
RUN ["gem", "install", "fluent-plugin-elasticsearch", "--no-document", "--version", "3.5.2"]
USER fluent

i run my Dockerfile

$ sudo docker build -t log-app .
$ sudo docker run -d --name logging log-app:latest
$ sudo docker logs -f logging

and I got result stuck, I don't know why

2020-10-26 10:24:58 +0000 [info]: parsing config file is succeeded path="/fluentd/etc/fluent.conf"
2020-10-26 10:24:58 +0000 [info]: gem 'fluent-plugin-elasticsearch' version '3.5.2'
2020-10-26 10:24:58 +0000 [info]: gem 'fluentd' version '1.11.4'
2020-10-26 10:24:58 +0000 [warn]: 'pos_file PATH' parameter is not set to a 'tail' source.
2020-10-26 10:24:58 +0000 [warn]: this parameter is highly recommended to save the position to resume tailing.
2020-10-26 10:24:58 +0000 [info]: using configuration file: <ROOT>
 <source>
   @type tail
   path "/var/log/logf/a.log"
   tag "test"
   read_from_head true
   <parse>
      @type "none"
      message_key "test"
      unmatched_lines
   </parse>
 </source>
 <match test>
    @type stdout
 </match>
</ROOT>
2020-10-26 10:24:58 +0000 [info]: starting fluentd-1.11.4 pid=6 ruby="2.6.6"
2020-10-26 10:24:58 +0000 [info]: spawn command to main:  cmdline=["/usr/local/bin/ruby", "-Eascii- 8bit:ascii-8bit", "/usr/local/bundle/bin/fluentd", "-c", "/fluentd/etc/fluent.conf", "-p", "/fluentd/plugins", "--under-supervisor"]
2020-10-26 10:24:59 +0000 [info]: adding match pattern="test" type="stdout"
2020-10-26 10:24:59 +0000 [info]: adding source type="tail"
2020-10-26 10:24:59 +0000 [warn]: #0 'pos_file PATH' parameter is not set to a 'tail' source.
2020-10-26 10:24:59 +0000 [warn]: #0 this parameter is highly recommended to save the position to resume tailing.
2020-10-26 10:24:59 +0000 [info]: #0 starting fluentd worker pid=15 ppid=6 worker=0
2020-10-26 10:24:59 +0000 [info]: #0 fluentd worker is now running worker=0

I think this is a permission problem, but I'm not sure because this Fluentd not throw an error, can you solve this problem guys?

[SOLVED] completely solved by mr karan shah's explanation

i was solved with docker-compose with mounting volume, below:

in file /opt/app/docker-compose.yaml

version: '2'

services:
  fluentd:
    build: .
    container_name: fl-logging
    volumes:
      - "./conf/:/fluentd/etc:ro"
      - "/var/log/logf:/var/log/logf"

and run the docker compose

 $ sudo docker-compose up -d --build

Upvotes: 1

Views: 3555

Answers (1)

karan shah
karan shah

Reputation: 370

The issue is that you have not mounted the local log files into the Fluentd container for it to be accessible.

Use a command like below.

sudo docker run -d --name logging -v PATHTOYOURLOGFILE:/var/log/logf/ log-app:latest

Read more about volumes here.

You can also use a docker-compose file like below

version: '2.2'
services:
 fluentd:
    build: ./opt/app/
    container_name: fl01
    volumes:
      - "/opt/app/conf/:/fluentd/etc/:ro"
      - "PATHTOYOURLOGFILE:/var/log/logf/"
    networks:
      - elastic
    ports:
      - "9880:9880"
networks:
  elastic:
    driver: bridge

Upvotes: 2

Related Questions