0

I am aware of kafka console producer command and it is quite handy. I want to know is there a way to know for sure which key will go to which partition when we have key as well? Assuming we have 10 partitions in kafka topic, how will producer decide to which partition the key will go?

I thought it might use key.toString.hashCode() % (num_of_partitons), but i don't think this is the way kafka console producer employs.

Can we check to which partition producer will send data to?

OneCricketeer
  • 126,858
  • 14
  • 92
  • 185
CuriousMind
  • 6,665
  • 12
  • 49
  • 95

1 Answers1

1

If the default partioner is used it will not hashcode the keys string value, instead it uses a Murmur2 hashing algorithm as seen in the org.apache.kafka.clients.producer.internals.DefaultPartitioner code:

Utils.toPositive(Utils.murmur2(keyBytes)) % numPartitions;

The implementation of the algorithm can be found here.

OneCricketeer
  • 126,858
  • 14
  • 92
  • 185
wgroleau
  • 103
  • 6