0

I tried googling and wiki'ing these questions but can't seem to find concrete answers. Most of what I found involved using proofs with the master theorem, but I'm hoping for something in plain English that can be more intuitively remembered. Also I am not in school and these questions are for interviewing.

MEMORY:

What exactly does it mean to determine big-o in terms of memory usage? For example, why is heapsort considered to run with O(1) memory when you have to store all n items? Is it because you are creating only one structure for the heap? Or is it because you know its size and so you can create it on the stack, which is always constant memory usage?

SPEED:

How is the creation of the heap done in O(n) time if adding elements is done in O(1) but percolating is done in O(logn)? Wouldn't that mean you do n inserts at O(1) making it O(n) and percolating after each insert is O(logn). So O(n) * O(logn) = O(nlogn) in total. I also noticed most implementations of heap sort use a heapify function instead of percolating to create the heap? Since heapify does n comparisons at O(logn) that would be O(nlogn) and with n inserts at O(1) we would get O(n) + O(nlogn) = O(nlogn)? Wouldn't the first approach yield better performance than the second with small n?

I kind of assumed this above, but is it true that doing an O(1) operation n times would result in O(n) time? Or does n * O(1) = O(1)?

mpellegr
  • 2,818
  • 2
  • 19
  • 36
  • 1
    The memory is typically stated as O(1) *additional* memory. I.e. constant memory not counting the objects being sorted themselves. The second question [is a duplicate](http://stackoverflow.com/questions/9755721/build-heap-complexity). – Raymond Chen Mar 28 '13 at 14:26
  • Why is it O(1) though? You need to allocate the memory for all n items in the heap, shouldn't that be O(n)? – mpellegr Mar 28 '13 at 14:35
  • Note the word "additional". The memory for the items themselves is not counted as part of "additional". Imagine somebody says "This car costs $20,000 and for an additional $1000 we will add GPS navigation." And then you say "How can it be only $2000? Just the car itself costs $20,000!" The $2000 is the *additional* charge; it doesn't count the base charge of $20,000 for the car itself. Similarly, the O(1) is the *additional* memory usage, not counting the memory for the items themselves. – Raymond Chen Mar 28 '13 at 14:38
  • Oh okay, so if we were to think in C++, the heap wouldn't reallocate n pointers and copy the data from the original items in to them, it would instead just set the pointer the heap uses to the pointer that already exists? – mpellegr Mar 28 '13 at 14:43
  • 1
    That's what it means when they say "in place". I think you should familiarize yourself with terminology before diving into algorithmic details. – Raymond Chen Mar 28 '13 at 15:32

1 Answers1

0

So I found some useful info about building a binary heap from wikipedia: http://en.wikipedia.org/wiki/Binary_heap#Building_a_heap.

I think my main source of confusion was how "inserting" into a heap is both O(1) and O(logn), even though the first shouldn't be called an insertion and maybe just a build step or something. So you wouldn't use heapify anymore after you've already created your heap, instead you'd use the O(logn) insertion method.

The method of adding items iteratively while maintaining the heap property runs in O(nlogn) and creating the heap without respecting the heap property, and then heapifying, actually runs in O(n), the reason which isn't very intuitive and requires a proof, so I was wrong about that.

The removal step to get the ordered items is the same cost, O(nlogn), after each method has a heap that respects the heap property.

So in the end you'd have either an O(1) + O(n) + O(nlogn) = O(nlogn) for the build heap method, and an O(nlogn) + O(nlogn) = O(nlogn) for the insertion method. Obviously the first is preferable, especially for small n.

mpellegr
  • 2,818
  • 2
  • 19
  • 36