CODEWITHSUNDEEP
ruby-on-railsdockerelasticsearchlogstashkibana
Jose A
Jose A

Reputation: 11129

Sending Logs to Logstash in ELK using Ruby on Rails Docker Image

I've been following these tutorials, repos, docs, and everything:

Well, you got the point. I have several more which I've omitted due to brevity

Unfortunately, either they're outdated, don't show the whole picture, or my config is bad (I think it's the latter one).

I honestly don't know if I'm missing anything.

I'm currently using a docker-compose version 3. I'm currently using this image (sebp/elk). I'm able to boot everything, and access Kibana, but I am not able to send the logs to Logstash so it gets processed and sent to ElasticSearch.

I've tried these approaches to no avail:

Inside application.rb

1) Use Lograge and send it to port 5044 (which apparently is the one which Logstash is listening to)

    config.lograge.enabled = true
    config.lograge.logger = LogStashLogger.new(type: :udp, host: 'localhost',  port: 5044)

2) Setting it to STDOUT, and let Docker process it as gelf and send it to Logstash:

logger           = ActiveSupport::Logger.new(STDOUT)
logger.formatter = config.log_formatter
config.logger    = ActiveSupport::TaggedLogging.new(logger)

And mapping it back to the compose file:

  rails_app:
      logging:
          driver: gelf
          options:
            gelf-address: "tcp://localhost:5044"

3) I've tried

Other Errors I've encountered:

Whenever I try to use the config.lograge.logger = LogStashLogger.new(type: :udp, host: 'localhost', port: 5044) I get: - Errno::EADDRNOTAVAIL - Cannot assign requested address - connect(2) for "localhost port 5044. Funny thing this error may disappear from time to time.

Other problem is that whenever I try to create a dummy log entry I receive a Elasticsearch Unreachable: [http://localhost:9200...] This is inside the container... I don't know if it can't connect because the URL is exposed outside from it... or there's another error. I can curl to localhost:9200 and receive a positive response.

I was checking the sebp/ELK image, and I see that it's using Filebeat. Could that be the reason why I am not able to send the logs?

I'd appreciate any kind of help!

Just in case, here's my docker-compose.yml file:

version: '3'
services:
  db:
    image: mariadb:10.3.5
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: "rootPassword"
      MYSQL_USER: "ruby"
      MYSQL_PASSWORD: "userPassword"
      MYSQL_DATABASE: "dev"
    ports:
    - "3306:3306"
    volumes:
      - db-data:/var/lib/mysql/data
      - ./db/rails_cprint.sql:/docker-entrypoint-initdb.d/rails_cprint.sql
  elk:
    image: sebp/elk:623
    ports:
      - 5601:5601
      - 9200:9200
      - 5044:5044
      - 2020:2020
    volumes:
      - elasticsearch:/var/lib/elasticsearch
  app:
    build: .
    depends_on:
      - db
      - elk
    environment:
      RAILS_ENV: development
      LOGSTASH_HOST: localhost
      SECRET_MYSQL_HOST: 'db'
      SECRET_MYSQL_DATABASE: 'dev'
      SECRET_MYSQL_USERNAME: 'ruby'
      SECRET_MYSQL_PASSWORD: 'userPassword'
    command: bash -c "rm -f tmp/pids/server.pid && bundle exec rails s -p 3001 -b '0.0.0.0'"
    stdin_open: true
    tty: true
    logging:
      driver: gelf
      options:
        gelf-address: "tcp://localhost:5044"
    links:
      - db
    volumes:
      - "./:/var/www/cprint"
    ports:
      - "3001:3001"
    expose:
      - "3001"
volumes:
  db-data:
    driver: local
  elasticsearch:
    driver: local

Upvotes: 2

Views: 2178

Answers (1)

Peter
Peter

Reputation: 1

Don't use "localhost" in your configs. In the docker network, the service name is the host name. Use "elk" instead, for example:

config.lograge.logger = LogStashLogger.new(type: :udp, host: 'elk',  port: 5044)

Upvotes: -1

Related Questions