CODEWITHSUNDEEP
loggingamazon-elastic-beanstalkceleryamazon-cloudwatch
jack.odonoghue
jack.odonoghue

Reputation: 41

Getting custom elastic beanstalk logs into cloudwatch group

Having issues and been unable to create a custom cloudwatch log group from .ebextentions/logs.config

Here are different files I have tried.

1

--- 
files: 
  /opt/elasticbeanstalk/tasks/bundlelogs.d/celery_logs.conf: 
    content: |-
        /var/log/celery_beat.stdout.log
        /var/log/celery_flower.stdout.log
        /var/log/celery_worker.stdout.log
        /var/log/faust_worker.stdout.log
    group: root
    mode: "000755"
    owner: root

2

--- 
files: 
  "/opt/elasticbeanstalk/config/private/logtasks/bundle/applogs.conf" :
    mode: "000755"
    owner: root
    group: root
    content: |
      /var/log/celery_beat.stdout.log
      /var/log/celery_flower.stdout.log
      /var/log/celery_worker.stdout.log
      /var/log/faust_worker.stdout.log

3

packages:
  yum:
    awslogs: []

files:
  "/etc/awslogs/awscli.conf" :
    mode: "000600"
    owner: root
    group: root
    content: |
      [plugins]
      cwlogs = cwlogs
      [default]
      region = `{"Ref":"AWS::Region"}`

  "/etc/awslogs/config/logs.conf" :
    mode: "000600"
    owner: root
    group: root
    content: |
      [/var/log/celery_beat.stdout.log]
      log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "/var/log/celery_beat.stdout.log"]]}`
      log_stream_name = {instance_id}
      file = /var/log/celery_beat.stdout.log

      [/var/log/celery_flower.stdout.log]
      log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "/var/log/celery_flower.stdout.log"]]}`
      log_stream_name = {instance_id}
      file = /var/log/celery_flower.stdout.log

      [/var/log/celery_worker.stdout.log]
      log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "/var/log/celery_worker.stdout.log"]]}`
      log_stream_name = {instance_id}
      file = /var/log/celery_worker.stdout.log

commands:
  "01":
    command: systemctl enable awslogsd.service
  "02":
    command: systemctl restart awslogsd

The logs are properly showing up in the files:

/var/log/celery_beat.stdout.log
/var/log/celery_flower.stdout.log
/var/log/celery_worker.stdout.log
/var/log/faust_worker.stdout.log

But no log group is being created with no logs being transferred to it.

I've tried 15 or more other similar configurations with no luck.

Upvotes: 4

Views: 1443

Answers (3)

Kirtan
Kirtan

Reputation: 11

Amazon Linux 2 dosent support /opt/elasticbeanstalk/ and awslogs anymore.

To configure the custom logs follow the below approach.

  1. Create a .platform/hooks/postdeploy/ in root of your application
  2. Create a shell script with the log paths as required.
  3. Place the shell script in /postdeploy directory with .sh extension
  4. In Elasticbeanstalk console configuration setup a environment variable as we will fetch this in our script later.
  5. You should see your custom logs getting streamed in cloudwatch.

#!/bin/bash
ENV_NAME=$(/opt/elasticbeanstalk/bin/get-config environment | jq -r '.ENV_NAME')

cat > /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d/file_beanstalk.json <<EOF
{
        "logs": {
                "logs_collected": {
                        "files": {
                                "collect_list": [
                                        {
                                                "file_path": "/var/log/eb-engine.log",
                                                "log_group_name": "/aws/elasticbeanstalk/${ENV_NAME}/var/log/eb-engine.log",
                                                "log_stream_name": "{instance_id}"
                                        }
                                        {
                                                "file_path": "/var/log/messages",
                                                "log_group_name": "/aws/elasticbeanstalk/${ENV_NAME}/var/log/messages",
                                                "log_stream_name": "{instance_id}"
                                        }
                                ]
                        }
                }
        }
}
EOF

sudo sudo systemctl restart amazon-cloudwatch-agent

Upvotes: 1

andylee
andylee

Reputation: 433

Latest EC 2 does not use awslogs any more. SSH into your instance and check if your instance is using latest cloudwatch agent by typing

sudo amazon-cloudwatch-agent-ctl -a status

if it is running , add following config file to your ebextensions.

container_commands:   01_append_logs_config:
    command: |
      amazon-cloudwatch-agent-ctl -a append-config -m ec2 -c file:/tmp/custom-cloudwatch-config.json -s

files:   "/tmp/custom-cloudwatch-config.json":
    mode: "000600"
    owner: root
    group: root
    content: |
      {
        "logs": {
          "logs_collected": {
            "files": {
              "collect_list": [
                {
                  "file_path": "/var/log/messages",
                  "log_group_name": "`{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/messages"]]}`",
                  "log_stream_name": "{instance_id}"
                },
                {
                  "file_path": "/var/log/cfn-init.log",
                  "log_group_name": "`{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/cfn-init.log"]]}`",
                  "log_stream_name": "{instance_id}"
                },
                {
                  "file_path": "/var/log/cfn-init-cmd.log",
                  "log_group_name": "`{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/cfn-init-cmd.log"]]}`",
                  "log_stream_name": "{instance_id}"
                },
                {
                  "file_path": "/var/log/dnf.log",
                  "log_group_name": "`{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/dnf.log"]]}`",
                  "log_stream_name": "{instance_id}"
                },
                {
                  "file_path": "/var/log/eb-cfn-init.log",
                  "log_group_name": "`{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/eb-cfn-init.log"]]}`",
                  "log_stream_name": "{instance_id}"
                },
                {
                  "file_path": "/var/log/celery_beat.stdout.log",
                  "log_group_name": "`{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/celery_beat.stdout.log"]]}`",
                  "log_stream_name": "{instance_id}"
                },
                {
                  "file_path": "/var/log/celery_worker.stdout.log",
                  "log_group_name": "`{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/celery_worker.stdout.log"]]}`",
                  "log_stream_name": "{instance_id}"
                },
                {
                  "file_path": "/var/log/secure",
                  "log_group_name": "`{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/secure"]]}`",
                  "log_stream_name": "{instance_id}"
                },
                {
                  "file_path": "/var/log/cron",
                  "log_group_name": "`{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/cron"]]}`",
                  "log_stream_name": "{instance_id}"
                }
              ]
            }
          }
        }
      }

check https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/install-CloudWatch-Agent-commandline-fleet.html for appending custom config files that will add custom log files.

config file generation is explained in https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/create-cloudwatch-agent-configuration-file-wizard.html. You can generate config files that fit your situation by running the wizard and change the content of the file below with the result.

https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/UseCloudWatchUnifiedAgent.html

lastly make sure your elasticbeanstalk iam profile has the access to write to cloudwatch logs in your iam profile

Upvotes: 4

vinhboy
vinhboy

Reputation: 8982

Someone left an answer but it got deleted so I'll go ahead and just leave it here for y'all.

I think the reason OP, and I, couldn't get it working is because we went down the wrong path by following this tutorial:

https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/using-features.logging.html

In this tutorial they teach you to use a conf file in /opt/elasticbeanstalk/tasks/, but this is only for exporting the logs into the Elastic Beanstalk web console, NOT into CloudWatch logs.

To export custom logs into CloudWatch logs you have to use this tutorial (which is option #3 in OP's question)

https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/AWSHowTo.cloudwatchlogs.html

It specifically has a section for "Custom log file streaming" -- which is done by using .ebextensions to install a package called awslogs and configuring it yourself. This process is way more involved than simply putting in a conf file but that's how it has to be done.

Be sure to also check the Elastic Beanstalk role to make sure it has permissions to interact with CloudWatch logs.

Upvotes: 1

Related Questions