CODEWITHSUNDEEP
javaapache-kafkadistributedtracezipkin
jpiechowka
jpiechowka

Reputation: 75

Zipkin, using existing libraries to handle tracing in microservices connected with Apache Kafka

I would like to implement tracing in my microservices architecture. I am using Apache Kafka as message broker and I am not using Spring Framework. Tracing is a new concept for me. At first I wanted to create my own implementation, but now I would like to use existing libraries. Brave looks like the one I will want to use. I would like to know if there are some guides, examples or docs on how to do this. Documentation on Github page is minimal, and I find it hard to start using Brave. Or maybe there is better library with proper documentation, that is easier to use. I will be looking at Apache HTrace because it looks promising. Some getting started guides will be nice.

Upvotes: 2

Views: 1206

Answers (2)

x-x
x-x

Reputation: 341

I meet the same problem too.Here is my solution, a less hacky way as above said.

ServerSpan serverSpan = brave.serverSpanThreadBinder().getCurrentServerSpan();
TraceHeader traceHeader = convert(serverSpan);

//in kafka producer,user KafkaTemplete to send
String wrapMsg = "wrap traceHeader with originMsg ";
kafkaTemplate.send(topic, wrapMsg).get(10, TimeUnit.SECONDS);// use synchronization


//then in kafka consumer
 ConsumerRecords<String, String> records = consumer.poll(5000);
 // for loop 
 for (ConsumerRecord<String, String> record : records) {
     String topic = record.topic();
     int partition = record.partition();
     long offset = record.offset();
     String val = record.value();
     //parse val to json
     Object warpObj = JSON.parseObject(val);
     TraceHeader traceHeader = warpObj.getTraceHeader();
     //then you can do something like this
     MyRequest myRequest = new MyRequest(traceHeader, "/esb/consumer", "POST");

     brave.serverRequestInterceptor().handle(new HttpServerRequestAdapter(new MyHttpServerRequest(myRequest), new DefaultSpanNameProvider()));

    //then some httprequest within brave-apache-http-interceptors
    //http.post(url,content)
 }

you must implements MyHttpServerRequest and MyRequest.It is easy,you just return something a span need,such as uri,header,method. This is a rough and ugly code example,just offer an idea.

Upvotes: 0

Adrian Cole
Adrian Cole

Reputation: 792

There are a bunch of ways to answer this, but I'll answer it from the "one-way" perspective. The short answer though, is I think you have to roll your own right now!

While Kafka can be used in many ways, it can be used as a transport for unidirectional single producer single consumer messages. This action is similar to normal one-way RPC, where you have a request, but no response.

In Zipkin, an RPC span is usually request-response. For example, you see timing of the client sending to the server, and also the way back to the client. One-way is where you leave out the other side. The span starts with a "cs" (client send) and ends with a "sr" (server received).

Mapping this to Kafka, you would mark client sent when you produce the message and server received when the consumer receives it.

The trick to Kafka is that there is no nice place to stuff the trace context. That's because unlike a lot of messaging systems, there are no headers in a Kafka message. Without a trace context, you don't know which trace (or span for that matter) you are completing!

The "hack" approach is to stuff trace identifiers as the message key. A less hacky way would be to coordinate a body wrapper which you can nest the trace context into.

Here's an example of the former:

https://gist.github.com/adriancole/76d94054b77e3be338bd75424ca8ba30

Upvotes: 2

Related Questions