NLog to ElasticSearch error - Action/metadata line [1] contains an unknown parameter [_type]

Question

I'm trying to use NLog to log to ElasticSearch which I have running in a container in docker desktop. But I can't see any logs in Kibana and the NLog internal log shows errors:

Error ElasticSearch: Server error: ServerError: 400Type: illegal_argument_exception Reason: "Action/metadata line [1] contains an unknown parameter [_type]"

Here's the docker compose file I'm using:

version: "3.8"
services:

  elasticsearch:
    container_name: elastic-search
    image: elasticsearch:8.5.1
    restart: "no"
    environment:
      - xpack.security.enabled=false
      - discovery.type=single-node
    ulimits:
      memlock:
        soft: -1 
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
    cap_add: 
      - IPC_LOCK
    ports:
      - "9200:9200"
    volumes:
      - C:\development\docker-volumes\elasticsearch:/usr/share/elasticsearch/data
    
  kibana:
    container_name: kibana
    image: kibana:8.4.3
    restart: "no"
    environment:
      SERVER_NAME: kibana
      ELASTICSEARCH_HOSTS: http://elasticsearch:9200
    ports:
      - "5601:5601"
    depends_on:
      - elasticsearch

I'm using the NLog nuget packages:

• NLog 5.0.5
• NLog.Extensions.Hosting 5.1.0
• NLog.Extensions.Logging 5.1.0
• NLog.Targets.ElasticSearch 7.7.0

My NLog config file looks like this:

<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="http://www.nlog-project.org/schemas/NLog.xsd NLog.xsd"
      autoReload="true"
      throwExceptions="false"
      internalLogLevel="Debug" internalLogFile="C:/development/logs/nlog-internal.log">

    <extensions>
        <add assembly="NLog.Targets.ElasticSearch"/>
    </extensions>
    
    <variable name="microservice" value="MyMicroervice"/>
    <variable name="component" value="IntegrationTests"/>
    
    <targets async="true">
   
        <target xsi:type="File" name="file" 
                fileName="C:/development/logs/${microservice}/${shortdate}.log"
                layout="${longdate} ${component} ${uppercase:${level}} ${message}" />
        
        <target name="elastic" xsi:type="BufferingWrapper" flushTimeout="5000">
            <target xsi:type="ElasticSearch" index="microservice-logs"
                    uri="http://localhost:9200"
                    includeAllProperties ="true" >
                <field name="Component" layout="${component}" />
                <field name="TimeStamp" layout="${longdate}" />
                <field name="Level" layout="${uppercase:${level}}" />
                <field name="LoggerName" layout="${logger}" />
                <field name="Message" layout="${message}" />
                <field name="error" layout="${exception:format=tostring}" />
            </target>
        </target>
    </targets>

    <rules>
        <logger name="*" minlevel="Debug" writeTo="file" />
        <logger name="*" minlevel="Debug" writeTo="elastic" />
    </rules>
</nlog>

It's an integration test project using xUnit and dependency injection. To the setup looks like this:

public void ConfigureHost(IHostBuilder hostBuilder)
{
    var environmentName = Environment.GetEnvironmentVariable("ASPNETCORE_ENVIRONMENT") ?? "Development";

    _configuration = new ConfigurationBuilder()
        .SetBasePath(Directory.GetCurrentDirectory())
        .AddJsonFile("appsettings.json")
        .AddJsonFile($"appsettings.{environmentName}.json")
        .Build();

    hostBuilder.ConfigureHostConfiguration(builder => builder.AddConfiguration(_configuration));
    hostBuilder.UseNLog();
}

It looks like NLog is still posting something (_type) that Elastic isn't expecting anymore and it doesn't like it. But I can't find how to fix the issue? Have I even understood the problem correctly?

I've found similar questions that point to something called fluentd and suggest changing it's configuration, but I'm not using fluentd. Unless it's a dependency of the NLog.Targets.ElasticSearch package?

How can I affect the "Action/metadata" that NLog posts to Elastic?

Answer 1

Elastic v6 requires DocumentType, Elastic v7 deprecates DocumentType, Elastic v8 prevents DocumentType.

You can configure NLog ElasticSearch to not include DocumentType, by explicit specifying documentType="":

<target xsi:type="ElasticSearch" index="microservice-logs"
        uri="http://localhost:9200"
        documentType=""
        includeAllProperties ="true" >

See also: https://github.com/markmcdowell/NLog.Targets.ElasticSearch/wiki