CODEWITHSUNDEEP
google-cloud-platformgoogle-cloud-datastoregoogle-cloud-runopen-telemetrygoogle-cloud-trace
Tom Keen
Tom Keen

Reputation: 210

Datastore grpc are not being traced in Cloud Run

I have an instrumentation for tracing my Python backend that uses NDB to interact with Datastore, I follow the classic way for exporting traces to Cloud Trace:

from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.propagate import set_global_textmap
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.instrumentation.grpc import GrpcInstrumentorClient
from opentelemetry.exporter.cloud_trace import CloudTraceSpanExporter
from opentelemetry.instrumentation.fastapi import FastAPIInstrumentor
from opentelemetry.propagators.cloud_trace_propagator import CloudTraceFormatPropagator
from fastapi import FastAPI
from google.cloud import ndb

app = FastAPI()

client = ndb.Client()

class MyEntity(ndb.Model):
    name = ndb.StringProperty()
    value = ndb.IntegerProperty()

@app.get("/entities")
def get_entities():
    with client.context():
        entities = MyEntity.query().fetch(20)
        results = [
            {
                'id': entity.key.id(),
                'name': entity.name,
                'value': entity.value
            } for entity in entities
        ]
    return results

tracer_provider = TracerProvider()
tracer_provider.add_span_processor(BatchSpanProcessor(CloudTraceSpanExporter()))
trace.set_tracer_provider(tracer_provider)
set_global_textmap(CloudTraceFormatPropagator())

FastAPIInstrumentor().instrument_app(app, excluded_urls=EXCLUDED_PATHS_REGEX)

grpc_client_instrumentor = GrpcInstrumentorClient()
grpc_client_instrumentor.instrument(
        tracer_provider=tracer_provider
    )

When I tested the instrumentation locally (using Datastore Emulator), I saw the spans for Datastore grpc calls:

datastore spans

with the proper attributes:

enter image description here

However after deploying to Cloud Run, when I check the traces for the requests I have sent, the trace shows only one span for the request with the latency, and nothing more.The inspection of the trace also says "Missing span ID ..." . What else is needed to make the grpc call from Datastore to be traced in Cloud Run?

Upvotes: 0

Views: 74

Answers (1)

Tom Keen
Tom Keen

Reputation: 210

It turns out that the issue was not code-related, but a Cloud Run configuration related. The service account for Cloud Run must have the role Cloud Trace Agent.

Upvotes: 0

Related Questions