static void DeliverCallback(rd_kafka_t *rk, void *payload, size_t len, rd_kafka_resp_err_t error_code, void *opaque, void *msg_opaque)
{
if (error_code != RD_KAFKA_RESP_ERR_NO_ERROR)
{
// todo resend the message
}
}
my dr_cb is like this. I know how to get the content of the failed-to-be-sent mesasge, but how do I get the topic? what's the most elegant way to do this?
The dr_cb interface is old and deprecated, you should use the richer dr_msg_cb instead, where you have access to all the rd_kafka_message_t fields, for your case the rkmessage->rkt for the topic.
But, you typically should not retry producing a message on failure in a delivery report, since librdkafka will have done all in its power to produce the message within the configured constraints (message.timeout.ms and retries), there isn't much value the application can add by retrying the same message again.
Instead configure message.timeout.ms to accommodate your business needs, answering the question "How long does it make sense to try producing this piece of data?", and set retries to its maximum value (since the number of retries is irrelevant from an application perspective).
If duplicates or message ordering is critical you should also look into using the idempotent producer (enable.idempotence=true).
And finally, librdkafka provides a message persistance indicator API (rd_kafka_message_status()) for each delivered or failed message, letting the application know if the message was..:
See https://github.com/edenhill/librdkafka/blob/master/INTRODUCTION.md#message-reliability for more information on message reliablity in librdkafka.
