2

Frequently when developing with MessageHub, I find that I want to purge my development data from a topic.

How can I purge a MessageHub topic?

This question is similar to Purge Kafka Queue but differs because that question is directed at apache kafka and I'm not sure if Message Hub supports the kafka command line tools.

Community
  • 1
  • 1
Chris Snow
  • 20,818
  • 29
  • 115
  • 263

1 Answers1

4

The only way to purge a Kafka topic from within Message Hub is to delete and recreate the topic. You can do this manually using the Web UI provided by the Message Hub service. Alternatively you can use the REST API for administering Kafka topics. The advantage of using the REST API is that it can be scripted.

The Message Hub REST API is documented in Swagger here: https://github.com/ibm-messaging/message-hub-docs/blob/master/kafka-administration-api/KafkaTopicManagement.yaml. If you are not a Swagger Guru then the REST call to delete is:

POST /admin/topics/<TOPICNAME>

You will need to specify your Message Hub API key (from VCAP_SERVICES) using the X-Auth-Token header to authenticate the request. So a sample curl implementation would look like:

curl -k -v -X DELETE -H 'Content-Type: application/json' -H 'Accept: */*' \
    -H 'X-Auth-Token: yourapikeyhere' \                                          
    https://admin-endpoint-goes-here/admin/topics/<TOPICNAME>

The one gotcha is that Kafka topic deletion is asynchronous. So before you can re-create the topic, you need to make sure that the deletion process for the original topic has completed. This can be achieved by polling the following endpoint until it returns a 404 (Not Found) status code:

GET /topics/<TOPICNAME>

(Again the X-Auth-Token header must be present). In curl:

curl -k -v -H -H 'Accept: application/json' \
    -H 'X-Auth-Token: yourapikeyhere' \
    https://admin-endpoint-goes-here/topics/<TOPICNAME>

To (re-)create a topic requires the following REST request (also with an X-Auth-Token):

POST /admin/topics

The body of the request contains a JSON document with parameters describing the topic to create. For example:

{                                                                                
  "name": "TOPICNAME",                                                        
  "partitions": 2                                                                
}

In curl this would would be: 

curl -k -v -H 'Content-Type: application/json' -H 'Accept: */*' \
    -H 'X-Auth-Token: yourapikeyhere' \
    -d '{ "name": "TOPICNAME", "partitions": 2 }' \
    https://admin-endpoint-goes-here/admin/topics
Adrian Preston
  • 136
  • 4