Eric Hirst
Eric Hirst

Reputation: 1161

AWS CloudWatch logging working from one machine, not another

I have 2 EC2 Windows 10 instances. Using C# and AWSSDK.CloudWatchLogs, I'm able to write to AWS CloudWatch from one of them, but not the other. I can duplicate this with the code below:

C# code:

using System;

namespace AllPurposeDuck
{
    class Program
    {
        private static Logger _log = LogManager.GetCurrentClassLogger();
        static void Main(string[] args)
        {
            try
            {
                _log.Info("Quack!");
                _log.Debug("Quack!!");
                _log.Trace("Quack!!!");
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.ToString());
            }
        }
    }
}

app.config:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <configSections>
    <section name="nlog" type="NLog.Config.ConfigSectionHandler, NLog" />
  </configSections>

  <nlog throwExceptions="true"
        internalLogFile="c:\temp\allpurposeducklog.txt"
        internalLogLevel="Info"
        xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <targets>
      <target name="console" xsi:type="Console" layout="${longdate} | ${level} | ${machinename}| ${message}" />
      <target name="aws" xsi:type="AWSTarget" logGroup="Ducks_and_stuff" region="us-west-2" layout="${machinename} ${longdate} | ${level} | ${message}" />
    </targets>
    <rules>
      <logger name="*" minlevel="Trace" writeTo="console, aws" />
    </rules>
  </nlog>

  <startup>
    <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.7.2" />
  </startup>
</configuration>

On the machine that otherwise silently fails, I notice a short pause and see the following warning in my c:\temp\allpurposeducklog.txt local log file:

2021-03-20 01:36:40.7679 Info Shutting down logging...
2021-03-20 01:36:42.2809 Warn Target flush timeout. One or more targets did not complete flush operation, skipping target close.
2021-03-20 01:36:42.2809 Info Logger has been shut down.

The other machine quacks happily to CloudWatch (both quack to the console).

The failing machine is in the same VPC, with the same IAM role, availability zone, and security groups as the working machine. Both machines are working properly with CloudWatch Agent, which monitoring Windows memory and triggering alerts when it gets too low.

I'm having a hard time narrowing it down any further than that. Ideas? Is there somewhere in AWS where the reason for the failure might be getting recorded? Thanks.

Upvotes: 2

Views: 1087

Answers (1)

Julian
Julian

Reputation: 36710

Because the program exits soon after the logging, NLog could still be busy with writing the logs. If the target use asynchronous logging, NLog needs to know it should wait on writing.

It's always recommend to give some time just before the program exit. Add at the end of Main:

LogManager.Shutdown();

This will give NLog 15 seconds at max.

If that isn't enough time, you could flush first and wait for example max 30 sec.

LogManager.Flush(TimeSpan.FromSeconds(30));
LogManager.Shutdown();

See also https://github.com/NLog/NLog/wiki/Logging-troubleshooting

Upvotes: 2

Related Questions