1

how can I check memory usage on specific topic's partitions in kafka? I tried this one "kafka-log-dirs.sh --describe...", but I'm not sure what output means:

                            "partition": "simpleTopic-8",
                            "size": 2752037376,
                            "offsetLag": 0,
                            "isFuture": false

What "size" means here? Is it a total memory to use on specific partition or memory usage? I'm asking, because I set config "retention.bytes=2147483648" to this topic, so if "size" is just memory usage, my retention does not work.

mike
  • 9,910
  • 3
  • 18
  • 43
Rapiernik
  • 61
  • 7

1 Answers1

1

The output in size is what you are looking for.

Your cleanup policy is still working fine although it exceeded the setting in your retention.bytes configuration. Keep in mind, that the LogCleaner will only remove data on old/inactive segments and retention.bytes does not provide a hard maximum. You can read more about this in another answer of mine on Data still remains in Kafka topic even after retention time/size

mike
  • 9,910
  • 3
  • 18
  • 43