2

I am trying to connect my MirroMaker Kafka 0.9 to the Kafka Brokers 0.10 (IBM Message Hub) without success. The links I have followed are the followings, but they are mostly for Kafka clients 0.10:

https://console.bluemix.net/docs/services/MessageHub/messagehub050.html#kafka_using https://console.bluemix.net/docs/services/MessageHub/messagehub063.html#kafka_connect

Do you know the steps for Kafka clients 0.9 and how to use the MessageHubLoginModule and the jaas creation?

UPDATE

After different tests, the solution works correctly.

Edoardo Basili
  • 99
  • 1
  • 9

2 Answers2

2

In order to connect to the IBM message hub with a cloudera mirror maker you must set within cloudera manager the property Source Kafka Cluster's Security Protocol: source.security.protocol as PLAINTEXT and pass the following properties as Kafka MirrorMaker Advanced Configuration Snippet (Safety Valve) for mirror_maker_consumers.properties: 

security.protocol=SASL_SSL
sasl.mechanism=PLAIN
ssl.protocol=TLSv1.2
ssl.enabled.protocols=TLSv1.2
ssl.endpoint.identification.algorithm=HTTPS

It worked for me.

Beniamino Del Pizzo
  • 763
  • 1
  • 6
  • 17
1

First you should not be building a new Message Hub application using Kafka 0.9.

We've deprecated the custom login module 0.9 requires and our newer clusters won't support it. You should be using a Kafka client >= 0.10.2 as they properly support Sasl Plain authentication which is required by Message Hub. The newer Kafka clients offer many more features and are just better.

In case you're absolutely stuck with 0.9, you need:

  • The following properties set:

    security.protocol=SASL_SSL
ssl.protocol=TLSv1.2
ssl.enabled.protocols=TLSv1.2

  • A JAAS file containing:

    KafkaClient {
    com.ibm.messagehub.login.MessageHubLoginModule required
    serviceName="kafka"
    username="<USERNAME>"
    password="<PASSWORD>";
};

  • The custom login module JAR in the path:

    The file is available on Github: https://github.com/ibm-messaging/message-hub-samples/blob/master/kafka-0.9/message-hub-login-library/messagehub.login-1.0.0.jar

  • The java.security.auth.login.config Java property set:

    It needs to point to your JAAS file and can be either:

    • in the java command line using -Djava.security.auth.login.config=<PATH TO JAAS> or
    • programmatically using System.setProperty("java.security.auth.login.config", "<PATH TO JAAS>");
Mickael Maison
  • 18,458
  • 7
  • 48
  • 49