CODEWITHSUNDEEP
kubernetestraceistio
arkadi4
arkadi4

Reputation: 71

x-b3-sampled header is always set to 0 when accessing service through ingress controller

I have Kubernetes 1.17.5 and Istio 1.6.8 installed with demo profile.

And here is my test setup [nginx-ingress-controller] -> [proxy<->ServiceA] -> [proxy<->ServiceB]

When I'm sending requests to ingress controller I can see that ServiceA receives all required tracing headers from the proxy

x-b3-traceid: d9bab9b4cdc8d0a7772e27bb7d15332f
x-request-id: 60e82827a270070cfbda38c6f30f478a
x-envoy-internal: true
x-b3-spanid: 772e27bb7d15332f
x-b3-sampled: 0
x-forwarded-proto: http

Problem is x-b3-sampled is always set to 0 and no spans/traces are getting pushed to Jaeger

Few things I've tried

  1. I've added Gateway and VirtualService to ServiceA to expose it through Istio ingressgateway. When I send traffic through ingressgateway everything works as expected. I can see traces [ingress-gateway]->[ServiceA]->[ServiceB] in the JaegerUI
  2. I've also tried to install Istio with custom config and play with tracing related parameters with no luck.

Here is the config I've tried to use

apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  meshConfig:
    enableTracing: true
    defaultConfig:
      tracing:
        sampling: 100
  addonComponents:
    tracing:
      enabled: true
    grafana:
      enabled: false
    istiocoredns:
      enabled: false
    kiali:
      enabled: false
    prometheus:
      enabled: false
  values:
    tracing:
      enabled: true
    pilot:
      traceSampling: 100

Upvotes: 4

Views: 3048

Answers (1)

arkadi4
arkadi4

Reputation: 71

After few days of digging I've figured it out. Problem is in the format of the x-request-id header that nginx ingress controller uses.

Envoy proxy expects it to be an UUID (e.g. x-request-id: 3e21578f-cd04-9246-aa50-67188d790051) but ingrex controller passes it as a non-formatted random string (x-request-id: 60e82827a270070cfbda38c6f30f478a). When I pass properly formatted x-request-id header in the request to ingress controller its getting passed down to envoy proxy and request is getting sampled as expected. I also tried to remove x-request-id header from the request from ingress controller to ServiceA with a simple EnvoyFilter. And it also works as expected. Envoy proxy generates a new x-request-id and request is getting traced.

Upvotes: 3

Related Questions