1
$ kafka-consumer-groups.sh  --bootstrap-server localhost:9092 --list
Note: This will not show information about old Zookeeper-based consumers.
$ kafka-consumer-groups.sh  --zookeeper localhost:2181  --list
Note: This will only show information about consumers that use ZooKeeper (not 
those using the Java consumer API).
console-consumer-13585
$

To be more precise, only one consumer group is listed, while the actual number seen via Kafka Tool is substantially bigger. How to fix that, apart from opening a Jira ticket with issues.apache.org? Maybe actually it is not a defect but I am doing something wrong? And when I try to describe the one that is listed, I get some error:

$ kafka-consumer-groups.sh  --zookeeper localhost:2181  --group console- 
consumer-13585 --describe
Note: This will only show information about consumers that use ZooKeeper 
(not those using the Java consumer API).
Error: Executing consumer group command failed due to 
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = 
NoNode for /consumers/console-consumer-13585/ids
$

while in Kafka Tool, the same consumer group is normally presented, without visible problems.

I see similar issue reported here by IanVaughan: https://github.com/kafkaex/kafka_ex/issues/311 but I don't see resolution. I am using kafka_2.11-1.1.0

So, this listing via kafka-consumer-groups.sh --bootstrap-server can somehow be compromised, so that the command doesn't return anything. Plus, the problem cannot be rectified easily by compacting or deleting log cleanup policies, because problematic records/messages in __consumer_offsets topic persevere. I don't see any resolution, anywhere on internet for such a problem. I don't know if the problem is only in persisting unnecessary and inactive consumer groups, or can it cause more serious problems. Therefore, I would kindly asked the moderator who marked this question as "already answered", to explain what answer are we talking about, or to remove that mark.

hdjur_jcv
  • 527
  • 4
  • 25
  • Reading this: https://stackoverflow.com/questions/29243896/removing-a-kafka-consumer-group-in-zookeeper , I realized how kafka-consumer-groups.sh works in the first case, when issued with option --bootstrap-server, it reads the __consumer_offsets topic (in the second case it reads zookeeper data), but obviously something prevents it from displaying the result, something that doesn't prevent Kafka Tool doing the same. I think it is connected to this problem: https://stackoverflow.com/questions/54670927/how-to-solve-a-problem-with-checkpointed-invalid-consumer-offsets-and-producer – hdjur_jcv Mar 07 '19 at 18:21
  • What answer is usable from "removing a kafka consumer group in zookeeper"? "How to solve a problem with checkpointed invalid __consumer_offsets and producer epoch on partitions of __transaction_state" is my question, and it definitely doesn't offer an answer to this question. – hdjur_jcv Mar 08 '19 at 13:16

0 Answers0