3

I'm trying to connect to a Message Hub Service. I can connect if I use the REST API but when I try to connect from my Java program there's always a timeout

I'm using the next configuration:

Properties producerProps = new Properties();
producerProps.put(SslConfigs.SSL_PROTOCOL_CONFIG, "TLSv1.2");
producerProps.put(SslConfigs.SSL_ENABLED_PROTOCOLS_CONFIG, "TLSv1.2");
producerProps.put(SslConfigs.SSL_TRUSTSTORE_LOCATION_CONFIG, "/Library/Java/JavaVirtualMachines/jdk1.8.0_66.jdk/Contents/Home/jre/lib/security/cacerts");
producerProps.put(SslConfigs.SSL_TRUSTSTORE_PASSWORD_CONFIG, "changeit");
producerProps.put(SslConfigs.SSL_TRUSTSTORE_TYPE_CONFIG, "JKS");
producerProps.put(SslConfigs.SSL_ENDPOINT_IDENTIFICATION_ALGORITHM_CONFIG, "HTTPS");
producerProps.put(CommonClientConfigs.SECURITY_PROTOCOL_CONFIG, "SSL");

producerProps.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, org.apache.kafka.common.serialization.ByteArraySerializer.class);
producerProps.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, org.apache.kafka.common.serialization.ByteArraySerializer.class);
producerProps.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "kafka01-prod01.messagehub.services.us-south.bluemix.net:9093,kafka02-prod01.messagehub.services.us-south.bluemix.net:9093,kafka03-prod01.messagehub.services.us-south.bluemix.net:9093,kafka04-prod01.messagehub.services.us-south.bluemix.net:9093,kafka05-prod01.messagehub.services.us-south.bluemix.net:9093");
producerProps.put(ProducerConfig.CLIENT_ID_CONFIG, "myApiKey");

KafkaProducer<byte[], byte[]> kafkaProducer = new KafkaProducer<>(producerProps);
ProducerRecord<byte[], byte[]> producerRecord = new ProducerRecord<>("myTopic", "records".getBytes(), "[{ \"value\" : \"test\" }]".getBytes());

RecordMetadata metadata = kafkaProducer.send(producerRecord).get();
System.out.println("Offset: " + metadata.offset());
kafkaProducer.close();

After a while the error is

Exception in thread "main" java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 231 ms.
at org.apache.kafka.clients.producer.KafkaProducer$FutureFailure.<init>(KafkaProducer.java:706)
at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:453)
at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:339)

I'm using kafka-clients version 0.9.0.0

Any thoughts?

Ruben J Garcia
  • 327
  • 4
  • 13
  • Let me see if I can reproduce this I'll post my findings later :) – Rodrigo Montano Dec 03 '15 at 21:11
  • I see an odd thing too. In the Bluemix Dashboard, when I open my Message Hub Service, there's no topics and there's a spinner with the message "Waiting for data..." but I'm able to create and consume topics with the rest API – Ruben J Garcia Dec 04 '15 at 09:15

3 Answers3

4

The exception you see is typically a symptom of the topic not having been created before use. Message Hub does not allow auto-creation of topics - they must be created either through the Bluemix Dashboard or programatically via the admin REST interface. When you got this error, had you created the topic first?

In case you hadn't found them already - here's a link to our samples, including a Java sample that creates a topic and then uses the Kafka Java client to produce/consume messages. https://github.com/ibm-messaging/message-hub-samples

Oliver Deakin
  • 209
  • 1
  • 8
  • I've created the topic via the REST API and I can send messages and consume then via REST API – Ruben J Garcia Dec 04 '15 at 10:38
  • Is it mandatory to use SASL? Or can I use SSL with my api key? About creating topic, I can't create any topic from the dashboard, the button is disabled – Ruben J Garcia Dec 04 '15 at 10:40
  • Ok, thanks for the update - I'll see if I can recreate the problem using your code snippet. SASL is not mandatory right now, but soon will be (before year end), so I'd recommend starting with SASL immediately if you can. The API key will still be used for the REST interface, but SASL will be the only authentication available when using the native Kafka interface. – Oliver Deakin Dec 04 '15 at 10:45
  • New update. I try with SASL and jaas.conf and the problem is the same, but I've downloaded the message-hub example and it works – Ruben J Garcia Dec 04 '15 at 11:19
1

Good news. Everything works now. It turns that the topic disappeared since the last time I used it

I did a POST request to the REST API to ensure that the topic was created and instead of receiving a 422 I received a 202 so the topic wasn't created. Now everything works like a charm

Thanks to everybody

Ruben J Garcia
  • 327
  • 4
  • 13
0

about the UI being stuck on "Waiting for data..." we're aware of the problem and working to fix it. cheers

Edoardo Comar
  • 511
  • 2
  • 5