8

I am having trouble understanding why solutions for finding the kth smallest element uses a Max heap approach. And for the kth largest element a min heap approach. Wouldn't it make more sense to use min heap to find kth smallest element, since the smallest element will always be root? So if we want to find the 3rd smallest element, then we just delete the root 2 times, build the heap, and we get the 3rd smallest. In a max heap the smallest is not at the root, so why is it better to use? The same goes for sorting in ascending or descending numbers in an array. I see most people use max heap for ascending.

1 Answers1

1

In fact, we can use both Min and Max heap to find the k-th smallest element:

  1. Just like you described, we can build a Min Heap, and then extract k-th element in a loop.

or

  1. We can build Max Heap with just k elements. Then we just compare the rest of the elements with the root, substituting with the root only elements which are smaller than the root, so the Max Heap has always k smallest elements.
Andriy Berestovskyy
  • 6,779
  • 2
  • 14
  • 29
  • But how would you know that you are at kth smallest element? Wouldn't you need to compare all elements in the array to the end before you know if it's the kth smallest? –  Nov 07 '18 at 16:39
  • @mth1417 sure, first you need to build a small Max Heap with first k elements, then we need to compare the rest of the array with Max Heap root. But to build Min Heap you need to compare all array elements as well. And on top of that, we then need to extract k elements, where k might be as big as n... – Andriy Berestovskyy Nov 07 '18 at 16:42
  • Why would you need to compare to all elements in a min heap if the smallest is always at the root, you would just have to check root? –  Nov 07 '18 at 16:43
  • 1
    @mth1417 we need to compare all elements in order to build the Min Heap. The complexity to build a heap is O(n * log n), so we touch all elements, and for each element, we do the heapify which is O(log n). For Max Heap we do heapify just for k elements, so it's O(k * log k) and then we just compare the rest of the array with the Max Heap root, occasionally calling heapify just for k elements. So complexity for Max Heap is O(n * log k), while for the Min Heap it's O(n * log n) + O(k * log n) to extract k min elements. – Andriy Berestovskyy Nov 07 '18 at 16:55
  • I am sorry but I can't see why the extraction is O(k*log n) when the extraction is just the first element always –  Nov 07 '18 at 16:59
  • One more moment: with Max Heap we can instantly keep k smallest elements in online mode - without having full array. – MBo Nov 07 '18 at 16:59
  • @mth1417 Extracting top causes rebuilding the heap in log(n) steps. Extracting k times - klogn – MBo Nov 07 '18 at 17:01
  • @mth1417 Because Heap looks like a Binary Search Tree, but in fact it's not. Every time we insert/extract an element, the heap must be "heapified", i.e. the properties of the heap must be restored. The complexity of "heapify" is O(log n). Please have a look on Wikipedia for more details or add another question if you still have some questions: https://en.wikipedia.org/wiki/Binary_heap – Andriy Berestovskyy Nov 07 '18 at 18:41
  • One small correction: heapify, e.g. constructing a heap from an arbitrary array, actually takes O(n) using the sift/bubble down approach. Source: https://stackoverflow.com/questions/9755721/how-can-building-a-heap-be-on-time-complexity (It's also noted in the Binary Heap wikipedia article) – KhalilRavanna Jul 17 '20 at 15:16
  • @KhalilRavanna Sorry, `heapify` is not a "constructing a heap from an arbitrary array". `Heapify` is a process to restore heap property, and it takes O(log n). Please read carefully the links, they all say the same... – Andriy Berestovskyy Jul 17 '20 at 19:57
  • Hmm I'm finding the terminology "heapify" in numerous places including the wikipedia article specifying what I described. Not great that the terminology seems to be mixed. Looks like you described exactly what you're talking about in your comment so please excuse my quibbling. https://en.wikipedia.org/wiki/Heap_(data_structure) – KhalilRavanna Jul 19 '20 at 02:33