CODEWITHSUNDEEP
javaapache-kafka
Bruno
Bruno

Reputation: 508

Learning Kafka 0.8.2

Having a heck of a time finding some simple getting started samples with the new release. Such things as KafkaProducer differ from the Producer examples and much of the older code on the web doesn't seem to compile the same.

Any guidance? The Apache Kafka site has zero examples of producers in Java.

Please advise.

Upvotes: 10

Views: 11756

Answers (4)

Kyr
Kyr

Reputation: 5491

It is always a good idea to check how the original authors are testing their code so you can get a feeling of what they are trying to achieve or the desired use (if and when there are tests provided :)

In this case, just check this code: https://github.com/apache/kafka/blob/0.8.2/examples/src/main/java/kafka/examples/Producer.java

:)

Upvotes: 0

halafi
halafi

Reputation: 1254

Try to have a look at the following example: https://github.com/CameronGregory/kafka/blob/master/TestProducer.java

Upvotes: 6

Minh-Triet LÊ
Minh-Triet LÊ

Reputation: 1394

In the example below, I create a producer using String as key and byte[] as message content.

Create a new producer using the essential parameters :

Properties props = new Properties();
props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "server1:123,server2:456");
props.put(ProducerConfig.RETRIES_CONFIG, "3");
props.put(ProducerConfig.ACKS_CONFIG, "all");
props.put(ProducerConfig.COMPRESSION_TYPE_CONFIG, "none");
props.put(ProducerConfig.BATCH_SIZE_CONFIG, 200);
props.put(ProducerConfig.BLOCK_ON_BUFFER_FULL_CONFIG, true);
props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringSerializer");
props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.ByteArraySerializer");

KafkaProducer<String, byte[]> producer = new KafkaProducer<String, byte[]>(props);

Synchronously send a message :

producer.send(new ProducerRecord<>(topic, msgKey, msgContent)).get();

Asynchronously send a message :

producer.send(new ProducerRecord<>(topic, msgKey, msgContent));

Your maven dependencies is good for consumer and producer. If you need only the producer, you can use :

<dependency>
    <groupId>org.apache.kafka</groupId>
    <artifactId>kafka-clients</artifactId>
    <version>0.8.2.0</version>
</dependency>

Be warned that the new Consumer API is available but not usable for now. In the code source, the new API will return null or throw exception.

Upvotes: 19

Bruno
Bruno

Reputation: 508

I had to regress because of the lack of good examples.

Here is part of my pom.xml

    <dependency>
            <groupId>org.apache.kafka</groupId>
            <artifactId>kafka_2.10</artifactId>
            <version>0.8.2.0</version>
            <scope>compile</scope>
    </dependency>

Here is my code -pending some testing.

    // KafkaProducer.java - A first pass to verify that we can bring in the appropriate
    //                   libraries using Maven

    // Supports unit tests
    package com.bruno;

    import org.junit.Test;
    import static org.junit.Assert.*;



    import java.util.Arrays;
    import java.util.Properties;

    import kafka.javaapi.producer.Producer;
    import kafka.producer.KeyedMessage;
    import kafka.producer.ProducerConfig;



    public class MyKafkaProducer
    {

        public static void main(String[] args) throws Exception {
            Properties props = new Properties();

            props.put("metadata.broker.list", "192.168.1.203:9092");
            props.put("serializer.class", "kafka.serializer.StringEncoder");
            props.put("request.required.acks", "1");
            ProducerConfig config = new ProducerConfig(props);

            Producer p = new Producer<String, String>(config);

            //sending...
            String topic = "test";
            String message = "Hello Kafka";
            KeyedMessage<String, String> keyedMessage = new KeyedMessage<String, String>(topic, message);
            p.send(keyedMessage);

        }

    }

Upvotes: -2

Related Questions