Laravel 5.6 aws cloudwatch log

Question

Upgraded laravel from 5.4 to 5.6. Laravel removed $app->configureMonologUsing since version 5.6

the tutorial from aws not applicable anymore. https://aws.amazon.com/tw/blogs/developer/php-application-logging-with-amazon-cloudwatch-logs-and-monolog/

anyone can advise me where to migrate the logic inside $app->configureMonologUsing ?

thanks

Answer 1

I would like to share my experience of implementing Cloudwatch for Laravel and Nginx logs. This guide will be relevant for Multicontainer Docker environment with the Elastic Beanstalk. My approach is related more to streaming logs instead of manually adding them.

Here are the steps:

1.

Add .ebextensions/1_cloudwatch.config file with such content:

option_settings:
  - namespace: aws:elasticbeanstalk:cloudwatch:logs
    option_name: StreamLogs
    value: true

This will turn on default instance log streaming (according to docs).

2.

Add .ebextensions/2_logs_streamtocloudwatch_linux.config file with such content:

###################################################################################################
#### The following file installs and configures the AWS CloudWatch Logs agent to push logs to a Log
#### Group in CloudWatch Logs.
#### 
#### The configuration below sets the logs to be pushed, the Log Group name to push the logs to and
#### the Log Stream name as the instance id. The following files are examples of logs that will be
#### streamed to CloudWatch Logs in near real time:
#### 
#### 
#### You can then access the CloudWatch Logs by accessing the AWS CloudWatch Console and clicking
#### the "Logs" link on the left. The Log Group name will follow this format:
####
#### /aws/elasticbeanstalk/<environment name>/<full log name path>
####
#### Please note this configuration can be used additionally to the "Log Streaming" feature:
#### http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/AWSHowTo.cloudwatchlogs.html
###################################################################################################

packages:
  yum:
    awslogs: []

files:
  "/etc/awslogs/awscli.conf" :
    mode: "000600"
    owner: root
    group: root
    content: |
      [plugins]
      cwlogs = cwlogs
      [default]
      region = `{"Ref":"AWS::Region"}`

  "/etc/awslogs/awslogs.conf" :
    mode: "000600"
    owner: root
    group: root
    content: |
      [general]
      state_file = /var/lib/awslogs/agent-state

  "/etc/awslogs/config/logs.conf" :
    mode: "000600"
    owner: root
    group: root
    content: |
      [/var/log/nginx/error]
      log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/nginx-error"]]}`
      log_stream_name = logs
      timestamp_format = '[%d/%b/%Y:%H:%M:%S %z]'
      file = /var/log/nginx/error.log
      buffer_duration = 5000
      use_gzip_http_content_encoding = true

      [/var/log/nginx/access]
      log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/nginx-access"]]}`
      log_stream_name = logs
      timestamp_format = '[%d/%b/%Y:%H:%M:%S %z]'
      file = /var/log/nginx/access.log
      buffer_duration = 5000
      use_gzip_http_content_encoding = true

      [/var/log/nginx/laravel]
      datetime_format = %Y-%m-%d %H:%M:%S
      log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/laravel"]]}`
      log_stream_name = logs
      timestamp_format = '[%Y-%m-%d %H:%M:%S]'
      file = /var/log/laravel/laravel-*.log
      buffer_duration = 5000
      use_gzip_http_content_encoding = true
      multi_line_start_pattern = {datetime_format}

commands:
  "01":
    command: chkconfig awslogs on
  "02":
    command: service awslogs restart

This script is provided by Amazon, here is the source - https://github.com/awsdocs/elastic-beanstalk-samples/blob/master/configuration-files/aws-provided/instance-configuration/logs-streamtocloudwatch-linux.config

The most interesting part here it's this one:

[/var/log/laravel]
datetime_format = %Y-%m-%d %H:%M:%S
log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/laravel"]]}`
log_stream_name = logs
timestamp_format = '[%Y-%m-%d %H:%M:%S]'
file = /var/log/laravel/laravel-*.log
buffer_duration = 5000
use_gzip_http_content_encoding = true
multi_line_start_pattern = {datetime_format}

These are our parameters for Laravel logging. Nothing special here.

• log_group_name - cloud watch log group name, you can set something simple here, like: production-laravel
• log_stream_name - name of group's stream, you can set any string here or you can pass these "variables": {instance_id}, {hostname}, {ip_address} instead of static string
• multi_line_start_pattern = {datetime_format} - we need this for preventing creating a new log record for each line from our log file

You can read more about these parameters in CloudWatch Agent reference - https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AgentReference.html

3.

We need to update our Dockerrun.aws.json.

In your volumes section, you'll need to add such items:

"volumes": [
    ...
    {
        "name": "awseb-logs-nginx-proxy",
        "host": {
            "sourcePath": "/var/log/nginx"
        }
    },
    {
        "name": "laravel-logs",
        "host": {
            "sourcePath": "/var/log/laravel"
        }
    }
    ...
]

Second, we need to update our containerDefinitions section and add mountPoints for a needed containers, like so:

"containerDefinitions": [
    ...
    {
        "name": "laravel-app",
        "image": "laravel-image",
        "hostname": "laravel",            
        "essential": true,
        "memory": 256,
        "mountPoints": [
            {
                "sourceVolume": "laravel-logs",
                "containerPath": "/var/www/storage/logs"
            }
        ]    
    },
    {
        "name": "nginx",
        "image": "nginx-image",
        "hostname": "nginx",
        "essential": true,
        "memory": 128,
        "volumesFrom": [
            {
                "sourceContainer": "app"
            }
        ],
        "portMappings": [
            {
                "hostPort": 80,
                "containerPort": 80
            }
        ],
        "mountPoints": [
            {
                "sourceVolume": "awseb-logs-nginx-proxy",
                "containerPath": "/var/log/nginx"
            }
        ],
        "links": ["laravel-app"]
    }
    ...
]

In result your Dockerrun.aws.json file may look something like this:

{
    "AWSEBDockerrunVersion": 2,
    "volumes": [
        {
            "name": "awseb-logs-nginx-proxy",
            "host": {
                "sourcePath": "/var/log/nginx"
            }
        },
        {
            "name": "laravel-logs",
            "host": {
                "sourcePath": "/var/log/laravel"
            }
        }
    ],
    "containerDefinitions": [
        {
            "name": "laravel-app",
            "image": "laravel-image",
            "hostname": "laravel",            
            "essential": true,
            "memory": 256,
            "mountPoints": [
                {
                    "sourceVolume": "laravel-logs",
                    "containerPath": "/var/www/storage/logs"
                }
            ] 
        },
        {
            "name": "nginx",
            "image": "nginx-image",
            "hostname": "nginx",
            "essential": true,
            "memory": 128,
            "volumesFrom": [
                {
                    "sourceContainer": "app"
                }
            ],
            "portMappings": [
                {
                    "hostPort": 80,
                    "containerPort": 80
                }
            ],
            "mountPoints": [
                {
                    "sourceVolume": "awseb-logs-nginx-proxy",
                    "containerPath": "/var/log/nginx"
                }
            ],
            "links": ["laravel-app"]
        }        
    ]    
}

4.

You need to check if your IAM instance profile role is allowed to add logs to CloudWatch. By default the role's name is aws-elasticbeanstalk-ec2-role, but you can check it on Configuration page of the environment, under Security section.

Here is the policy you need to add (source):

{
"Version": "2012-10-17",
"Statement": [
  {
    "Effect": "Allow",
    "Action": [
      "logs:CreateLogGroup",
      "logs:CreateLogStream",
      "logs:PutLogEvents",
      "logs:DescribeLogStreams"
    ],
    "Resource": [
      "arn:aws:logs:*:*:*"
    ]
  }
 ]
}

Or you can add CloudWatchLogsFullAccess policy just for testing.

---

In result, you'll receive Log groups like these:

• /aws/elasticbeanstalk/your-env-name/var/log/laravel
• /aws/elasticbeanstalk/your-env-name/var/log/nginx-access
• /aws/elasticbeanstalk/your-env-name/var/log/nginx-error

Answer 2

IF you are using Laravel on AWS ECS.

Inside .env file add this LOG_CHANNEL=stderr

It will write logs to CloudWatch when you configure the Task Definition.

Answer 3

If you are running on AWS EC2 instances and log a lot of info / debug messages sending logs real time can slow down your application response times. Instead you can have CloudWatch agent watching your laravel.log to send new log entries in i.e. every 5 seconds. CloudWatch agent can ship all your system logs and any system metrics like CPU so you can create cloud watch alerts.

CloudWatch Agent

Answer 4

Install the latest version of CloudWatch handler library with:

composer require maxbanton/cwh

You can add a custom channel in config/logging.php like:

'cloudwatch' => [
  'driver' => 'custom',
  'via' => \App\Logging\CloudWatchLoggerFactory::class,
  'sdk' => [
    'region' => env('AWS_DEFAULT_REGION', 'us-east-1'),
    'version' => 'latest',
    'credentials' => [
      'key' => env('AWS_ACCESS_KEY_ID'),
      'secret' => env('AWS_SECRET_ACCESS_KEY')
    ]
  ],
  'retention' => env('CLOUDWATCH_LOG_RETENTION',7),
  'level' => env('CLOUDWATCH_LOG_LEVEL','error')
],

and a factory class App/Logging/CloudWatchLoggerFactory.php as:

<?php

namespace App\Logging;

use Aws\CloudWatchLogs\CloudWatchLogsClient;
use Maxbanton\Cwh\Handler\CloudWatch;
use Monolog\Logger;

class CloudWatchLoggerFactory
{
    /**
     * Create a custom Monolog instance.
     *
     * @param  array  $config
     * @return \Monolog\Logger
     */
    public function __invoke(array $config)
    {
        $sdkParams = $config["sdk"];
        $tags = $config["tags"] ?? [ ];
        $name = $config["name"] ?? 'cloudwatch';

        // Instantiate AWS SDK CloudWatch Logs Client
        $client = new CloudWatchLogsClient($sdkParams);

        // Log group name, will be created if none
        $groupName = config('app.name') . '-' . config('app.env');

        // Log stream name, will be created if none
        $streamName = config('app.hostname');

        // Days to keep logs, 14 by default. Set to `null` to allow indefinite retention.
        $retentionDays = $config["retention"];

        // Instantiate handler (tags are optional)
        $handler = new CloudWatch($client, $groupName, $streamName, $retentionDays, 10000, $tags);

        // Create a log channel
        $logger = new Logger($name);
        // Set handler
        $logger->pushHandler($handler);

        return $logger;
    }
}