Application insights end to end transaction details does not show whole sequence of telemetry

Question

We are having multiple microservices and we have application insights enabled on all of them.When i see end to end transaction details for a scenario like UI calls Service A which does some db operations and then calls service B which inturn does some db operations and calls service C. In this kind of scenario my end to end trasaction only shows me first level of calls (i.e. Service A , db operations and call to service B) and after that just says "More details for this call are not available due to sampling".

I have even tried removing "AdaptiveSamplingTelemetryProcessor" from app insights config but still the same thing. Below is the screenshot.

Answer 1

While the message on screen only mentions sampling, it more generically means that within the next microservice's telemetry, nothing was found with the same operation id - but that it found reason to expect that there should be telemetry.

The fix for this is to ensure that operation id (and parent id) is being propagated to all microservices and is used in the telemetry recorded by each microservice.

Typically, when using the latest Application Insights SDKs, service-to-service calls over HTTP are bound to share an operation id via correlation headers. These headers exist on the request to an instrumented microservice and that service's response back. The existence of these headers establish that "reason to expect that there should be telemetry".

Because the portal found evidence those headers existed but did not find telemetry from the next service (and sampling is disabled), it looks like that telemetry has been recorded with a different operation id in the next service. The additional library you're using (owinextensions) seems to overwrite operation id with its own generation and instructs to remove the built-in initializers that the SDK uses to make correlation happen.

To implement the fix bolded above in this situation, I would either:

• Find a way to go without owinextensions and restore the default initializers
• Use owinextensions' separate operation id persistence headers
• Combine the documented sample code for reading default correlation headers in OWIN with owinextensions' ability to change its operation id generation.