Python confluent_kafka: consume(0) cannot trigger callbacks

Question

I am using confluent-kafka-python and librdkafka.

But I think, confluent_kafka.Consumer.consume(0) cannot trigger stats_cb.

Consumer_consume(0) => rd_kafka_consume_batch_queue => rd_kafka_q_serve_rkmessages

In, rd_kafka_poll_cb is called in the while (cnt < rkmessages_size) loop. If rkmessages_size is 0, it is not called.

In this case, rk_rep may become larger and larger.

How to trigger stats_cb in Consumer_consume(0)? Thanks

Answer 1

Callbacks are served from rd_kafka_q_serve_rkmessages(): https://github.com/edenhill/librdkafka/blob/b50e247b77864c595569f7f7e9ec8dde63258210/src/rdkafka_queue.c#L574

If you're not seeing your stats_cb being triggered, make sure that statistics.interval.ms is set to a non-zero value.