CODEWITHSUNDEEP
.net-6.0open-telemetry
Mihai
Mihai

Reputation: 2760

OpenTelemetry .NET 6 OTPL Exporter exception

I have an ASPNET 6 application to which I've added OpenTelemetry with the OTPL Exporter configured as follows:

services.AddOpenTelemetryTracing(tracerProviderBuilder =>
        {
            tracerProviderBuilder
                .AddOtlpExporter(opt =>
                {
                    opt.Protocol = OtlpExportProtocol.Grpc;
                    opt.BatchExportProcessorOptions = new BatchExportProcessorOptions<Activity>
                    {
                        ////MaxExportBatchSize = 16 // Set a low batch size
                    };
                })
                .AddSource(serviceName)
                .AddSource("MySqlConnector")        // Or remove this
                .SetResourceBuilder(
                    ResourceBuilder.CreateDefault()
                        .AddService(serviceName: serviceName, serviceVersion: serviceVersion)
                        .AddDetector(new AWSEC2ResourceDetector()))
                .AddAWSInstrumentation()
                .AddHttpClientInstrumentation()
                .AddAspNetCoreInstrumentation()
                .AddRedisInstrumentation();
        });

And the following packages/versions

<PackageReference Include="OpenTelemetry" Version="1.4.0-alpha.2" />
<PackageReference Include="OpenTelemetry.Contrib.Extensions.AWSXRay" Version="1.2.0" />
<PackageReference Include="OpenTelemetry.Contrib.Instrumentation.AWS" Version="1.0.1" />
<PackageReference Include="OpenTelemetry.Exporter.OpenTelemetryProtocol" Version="1.3.0" />
<PackageReference Include="OpenTelemetry.Extensions.Hosting" Version="1.0.0-rc9.6" />
<PackageReference Include="OpenTelemetry.Instrumentation.AspNetCore" Version="1.0.0-rc9.6" />
<PackageReference Include="OpenTelemetry.Instrumentation.Http" Version="1.0.0-rc9.6" />
<PackageReference Include="OpenTelemetry.Instrumentation.StackExchangeRedis" Version="1.0.0-rc9.7" />

I have a bunch of integration tests that fail when I run them with the following exception

Exit code is -1073741819 (Fatal error. System.AccessViolationException: Attempted to 
read or write protected memory. This is often an indication that other memory is 
corrupt.
   at System.Runtime.CompilerServices.CastHelpers.IsInstanceOfClass(Void*, System.Object)
   at OpenTelemetry.Internal.TagTransformer`1[[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].TryTransformTag(System.Collections.Generic.KeyValuePair`2<System.String,System.Object>, System.__Canon ByRef)
   at OpenTelemetry.Exporter.OpenTelemetryProtocol.Implementation.ActivityExtensions+TagEnumerationState.ForEach(System.Collections.Generic.KeyValuePair`2<System.String,System.Object>)
   at DynamicClass.AllocationFreeForEachDelegate(System.Collections.Generic.IEnumerable`1<System.Collections.Generic.KeyValuePair`2<System.String,System.Object>>, TagEnumerationState ByRef, ForEachDelegate<System.Collections.Generic.IEnumerable`1<System.Collections.Generic.KeyValuePair`2<System.String,System.Object>>,System.Collections.Generic.KeyValuePair`2<System.String,System.Object>,TagEnumerationState>)
   at OpenTelemetry.Trace.ActivityHelperExtensions+ActivityTagsEnumeratorFactory`1[[OpenTelemetry.Exporter.OpenTelemetryProtocol.Implementation.ActivityExtensions+TagEnumerationState, OpenTelemetry.Exporter.OpenTelemetryProtocol, Version=1.0.0.0, Culture=neutral, PublicKeyToken=7bd6737fe5b67e3c]].Enumerate(System.Diagnostics.ActivityEvent, TagEnumerationState ByRef)
   at OpenTelemetry.Trace.ActivityHelperExtensions.EnumerateTags[[OpenTelemetry.Exporter.OpenTelemetryProtocol.Implementation.ActivityExtensions+TagEnumerationState, OpenTelemetry.Exporter.OpenTelemetryProtocol, Version=1.0.0.0, Culture=neutral, PublicKeyToken=7bd6737fe5b67e3c]](System.Diagnostics.ActivityEvent, TagEnumerationState ByRef)
   at OpenTelemetry.Exporter.OpenTelemetryProtocol.Implementation.ActivityExtensions.ToOtlpEvent(System.Diagnostics.ActivityEvent)
   at OpenTelemetry.Exporter.OpenTelemetryProtocol.Implementation.ActivityExtensions+EventEnumerationState.ForEach(System.Diagnostics.ActivityEvent)
   at OpenTelemetry.Trace.ActivityHelperExtensions+ActivityEventsEnumeratorFactory`1[[OpenTelemetry.Exporter.OpenTelemetryProtocol.Implementation.ActivityExtensions+EventEnumerationState, OpenTelemetry.Exporter.OpenTelemetryProtocol, Version=1.0.0.0, Culture=neutral, PublicKeyToken=7bd6737fe5b67e3c]].ForEachEventCallback(EventEnumerationState ByRef, System.Diagnostics.ActivityEvent)
   at DynamicClass.AllocationFreeForEachDelegate(System.Collections.Generic.IEnumerable`1<System.Diagnostics.ActivityEvent>, EventEnumerationState ByRef, ForEachDelegate<System.Collections.Generic.IEnumerable`1<System.Diagnostics.ActivityEvent>,System.Diagnostics.ActivityEvent,EventEnumerationState>)
   at OpenTelemetry.Exporter.OpenTelemetryProtocol.Implementation.ActivityExtensions.AddBatch(Opentelemetry.Proto.Collector.Trace.V1.ExportTraceServiceRequest, Opentelemetry.Proto.Resource.V1.Resource, OpenTelemetry.Batch`1<System.Diagnostics.Activity> ByRef)
   at OpenTelemetry.Exporter.OtlpTraceExporter.Export(OpenTelemetry.Batch`1<System.Diagnostics.Activity> ByRef)
   at OpenTelemetry.BatchExportProcessor`1[[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].ExporterProc()
   at System.Threading.Thread.StartCallback())

This scenario does not reproduce if I do one of two things

Any ideas?

Thanks

Upvotes: 0

Views: 1172

Answers (1)

Mihai
Mihai

Reputation: 2760

It appears that a bug has already been logged and a fix is expected in 1.3.1 https://github.com/open-telemetry/opentelemetry-dotnet/issues/3593

Upvotes: 1

Related Questions