How to stream tomcat catalinat.out logs to cloud watch?

Question

I want to stream tomcat catalinat.out logs to cloud watch:

This is my config I follow:

https://github.com/awsdocs/elastic-beanstalk-samples/blob/master/configuration-files/aws-provided/instance-configuration/logs-streamtocloudwatch-linux.config

But I don't see catalina.out in cloudwatch console :

This is error , I have in awslogs.log How can I solve it. 2020-05-22 18:15:55,450 - cwlogs.push.batch - WARNING - 3374 - Thread-29 - CreateLogGroup failed with exception An error occurred (AccessDeniedException) when calling the CreateLogGroup operation: User: arn:aws:sts::610232524349:assumed-role/aws-elasticbeanstalk-ec2-role/i-099300c0bfd4b6a28 is not authorized to perform: logs:CreateLogGroup on resource: arn:aws:logs:eu-central-1:610232524349:log-group:/aws/elasticbeanstalk/************/var/log/tomcat8/catalina.out:log-stream:

Answer 1

I feel your pain! I've detailed in a new Medium blog how this all works and an example .ebextensions file and where to put it.

Below is an excerpt that you might be able to use, though the article explains how to determine the right folder/file(s) to stream.

packages:
  yum:
    awslogs: []

option_settings:
  - namespace: aws:elasticbeanstalk:cloudwatch:logs
    option_name: StreamLogs
    value: true
  - namespace: aws:elasticbeanstalk:cloudwatch:logs
    option_name: DeleteOnTerminate
    value: false
  - namespace: aws:elasticbeanstalk:cloudwatch:logs
    option_name: RetentionInDays
    value: 90

files:
  "/etc/awslogs/awscli.conf" :
    mode: "000600"
    owner: root
    group: root
    content: |
      [plugins]
      cwlogs = cwlogs
      [default]
      region = `{"Ref":"AWS::Region"}`

  "/etc/awslogs/config/logs.conf" :
    mode: "000600"
    owner: root
    group: root
    content: |
      [/var/log/tomcat/localhost.log]
      log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/tomcat/localhost.log"]]}`
      log_stream_name = {instance_id}
      file = /var/log/tomcat/localhost.*

      [/var/log/tomcat/catalina.log]
      log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/tomcat/catalina.log"]]}`
      log_stream_name = {instance_id}
      file = /var/log/tomcat/catalina.*

      [/var/log/tomcat/localhost_access_log.txt]
      log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/tomcat/access_log"]]}`
      log_stream_name = {instance_id}
      file = /var/log/tomcat/access_log.*

commands:
  "01":
    command: systemctl enable awslogsd.service
  "02":
    command: systemctl restart awslogsd

Answer 2

Steps to publish tomcat logs (catalina.out) to the CloudWatch stream

1. Create a new policy for EC2 to use AWS CloudWatch, providing access to create log groups, log streams and publish logs

       {
           "Version": "2012-10-17",
           "Statement": [
               {
                   "Sid": "VisualEditor0",
                   "Effect": "Allow",
                   "Action": "logs:*",
                   "Resource": "*"
               }
           ]
       }
   

2. Attach the policy to newly created or existing IAM role of the EC2 instance

3. Connect to the instance using SSH, using the PEM/PPK file.

4. Locate the AWS CloudWatch Logs Agent configuration file

       [ec2-user@ elasticbeanstalk]$ sudo su
       [root@ elasticbeanstalk]# find / -name "*awslogs.conf"
       /etc/awslogs/awslogs.conf
   

5. Edit the configuration file and add the entry for a log stream for tomcat logs. I have used catalina.out

       [ec2-user@ elasticbeanstalk]$ cat /etc/awslogs/awslogs.conf
       [general]
       state_file = /var/lib/awslogs/agent-state
   
       [tomcatLogs]
       log_group_name = tomcatLogs
       log_stream_name = catalinaLogs
       time_zone = LOCAL
       file = /[your-path-to]/tomcat8/catalina.out
       [ec2-user@ elasticbeanstalk]$
   

6. Restart the service AWS Logs

       [ec2-user@ elasticbeanstalk]$ sudo service awslogs restart
   

7. Revisit the CloudWatch log groups page, where you can see the new group is created with the name “tomcatLogs” and a log stream with the name “catalinaLogs”

Answer 3

With the sample provided you are not exporting the catalinat.out you are streaming to cloudwatch the following files:

• /var/log/dmesg
• /var/log/messages

To stream the catalitat.out you have to add the file to the configuration with the location of the log at the end of content section (Lines 61-71 on the sample provided)

It should be something like this replacing /path/to/catalitat.log with the actual path to the log:

[/path/to/catalitat.log]
log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "/path/to/catalitat.log"]]}`
log_stream_name = {instance_id}
file = /path/to/catalitat.log