messagehub104.md

copyright	lastupdated
years 2015, 2017	2017-08-03

{:new_window: target="_blank"} {:shortdesc: .shortdesc} {:screen: .screen} {:codeblock: .codeblock} {:pre: .pre}

Using the Kafka Java client

{: #kafka_using}

How to use, download, and run the Java Kafka API sample

{: notoc}

The Java™ Kafka API sample is an example producer and consumer that is written in Java, which directly uses the Kafka API. You can run this sample locally or in {{site.data.keyword.Bluemix_short}}.

The sample code is in the message-hub-samples GitHub project {:new_window}. Although the sample uses the Kafka API to send and receive messages, the sample uses the {{site.data.keyword.messagehub}} Administration API to create the topic it sends messages to and receives messages from.

For more information about the setup and running of the sample, see the README.md {:new_window}.

For a detailed walkthrough of how to run the sample, see Getting started with {{site.data.keyword.messagehub}}.

How to use, download, and run the Liberty for Java sample

{: #liberty_sample notoc}

The Liberty for Java sample implements a simple application that is deployed onto the Liberty runtime. The application uses the Kafka API for {{site.data.keyword.messagehub}} to produce and consume messages. The application also serves up a web front end that you can use for administration.

You can find the sample code in the message-hub-samples GitHub project {:new_window}.

How to migrate the Kafka client from 0.9.x to 0.10.x

{: #kafka_migrate notoc}

If you are using the Java clients, you can now use the publicly available 0.10.x Kafka clients. You are strongly encouraged to move from 0.9.x to the latest 0.10.x version (currently 0.10.2.1). Complete the following steps:

1. Delete the {{site.data.keyword.messagehub}} login jar module.

2. Change your jaas.conf file to the following:

       KafkaClient {
         org.apache.kafka.common.security.plain.PlainLoginModule required
         serviceName="kafka"
           username="<your username>"
           password="<your password>";
       };
   

   {: codeblock}

3. Add this line to your consumer and producer properties: sasl.mechanism=PLAIN

Using the sasl.jaas.config property

{: #sasl_prop notoc} If you're using a Kafka client at 0.10.2.1 or later, you can use the sasl.jaas.config property for client configuration instead of a JAAS file. To connect to {{site.data.keyword.messagehub}}, set sasl.jaas.config as follows:

    sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
    username="USERNAME" \
    password="PASSWORD";

{:codeblock}

where USERNAME and PASSWORD are the values from your {{site.data.keyword.messagehub}} Service Credentials tab in {{site.data.keyword.Bluemix_notm}}.

If you use sasl.jaas.config, clients running in the same JVM can use different credentials. For more information, see Configuring Kafka clients {:new_window}

APIs for topic administration

{: #topic_admin notoc}

If you're using a Kafka client at 0.11 or later, or Kafka Streams at 0.10.2.0 or later, you can use APIs to create and delete topics. We've put some restrictions on the settings allowed when you create topics. Currently, you can modify the following settings only:

cleanup.policy

Set to delete (default), compact or delete,compact

retention.ms

The default retention period is 24 hours. The minimum is 1 hour and the maximum is 30 days. Specify this value as multiples of hours.

**Note:** If the cleanup policy is compact only, we automatically add delete but disable deletion based on time. Messages in the topic are compacted up to 1 GB before being deleted.

Support for Kafka Streams

{: #kafka_streams notoc}

Starting from the Streams library 0.10.2.0, the topic APIs now work with {{site.data.keyword.messagehub}} with no setup required. Specify your SASL credentials using sasl.jaas.config or a JAAS file and set replication.factor to 3.

For example:

    props.put(StreamsConfig.REPLICATION_FACTOR_CONFIG, "3");
    props.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, BOOTSTRAP_SERVERS);
    props.put("security.protocol","SASL_SSL");
    props.put("sasl.mechanism","PLAIN");
    props.put("ssl.protocol","TLSv1.2");
    props.put("ssl.enabled.protocols","TLSv1.2");
    props.put("sasl.jaas.config","org.apache.kafka.common.security.plain.PlainLoginModule required username=\"USERNAME\" password=\"PASSWORD\";");

{:codeblock}

where BOOTSTRAP_SERVERS, USERNAME, and PASSWORD are the values from your {{site.data.keyword.messagehub}} Service Credentials tab in {{site.data.keyword.Bluemix_notm}}.