19

Wikipedia lists the median-of-medians algorithm as requiring O(1) auxiliary space.

However, in the middle of the algorithm, we make a recursive call on a subarray of size n/5 to find the median of medians. When this recursive call returns, we use the returned median of medians as a pivot to partition the array.

Doesn't this algorithm push O(lg n) activation records onto the run-time stack as a part of the recursion? From what I can see, these recursive calls to find successive medians of medians cannot be tail-call optimized because we do extra work after the recursive calls return. Therefore, it seems like this algorithm requires O(lg n) auxiliary space (just like Quicksort, which Wikipedia lists as requiring O(lg n) auxiliary space due the space used by the run-time stack).

Am I missing something, or is the Wikipedia article wrong?

(Note: The recursive call I'm referring to is return select(list, left, left + ceil((right - left) / 5) - 1, left + (right - left)/10) on the Wikipedia page.)

0x90
  • 34,073
  • 33
  • 137
  • 218
John Kurlak
  • 6,070
  • 7
  • 38
  • 56
  • That's just an example of how it's used in quickselect. If you read the article closely, you'll find that *only* the pivot function contains the actual median-of-medians algorithm. – Nuclearman Jan 02 '16 at 23:46
  • 1
    @Nuclearman That's a fair point, but the `pivot` function makes a call to `select,` so we can't discount the space required for `select`. The Wikipedia article describes the two functions as *mutually recursive*. If we ignore the call to `select`, we don't end up with a median of medians. Instead, we end up with `n/5` medians of 5. – John Kurlak Jan 03 '16 at 01:17
  • After a bit more thought, it seems like Wikipedia may not have the most space efficient version listed. The reason it's not using space for the stack is that it doesn't need one, if you convert it from recursive to iterative. Take a look at the iterative versions of quickselect and you'll notice there's no stack, as it can be done just with loops. However, quicksort does need a stack (implicitly or explicitly). – Nuclearman Jan 03 '16 at 02:58
  • 1
    @Nuclearman Quickselect doesn't need a stack because it can be tail-call optimized. It can be tail-call optimized because we're essentially only traversing one root-to-leaf path in our recursion. We don't need to remember our history. Median-of-medians cannot be optimized in the same way because it has multiple recursive calls per level. In other words, we traverse multiple root-to-leaf paths during the execution of the algorithm. We need to remember our history so that we know where to go back when we reach a leaf. We cannot do such a traversal in constant space. – John Kurlak Jan 03 '16 at 03:10
  • 2
    @John I'm currently working on this as well and agree with you on everything. I [asked the author](https://en.wikipedia.org/wiki/User_talk:Sligocki#Median_of_medians_with_O.281.29_auxiliary_space.3F) of wikipedia's O(1) claim, hopefully we'll get an answer. – Stefan Pochmann Jan 03 '16 at 03:41
  • 1
    @StefanPochmann Thanks Stefan. I first started investigating this based on Wiggle Sort II from Leetcode, which you also appear to be looking at :) Thanks for reaching out to the original author. Didn't know that was an option! – John Kurlak Jan 03 '16 at 03:46
  • 1
    @John Yeah, leetcode as well. I was almost sure it wasn't a coincidence. But it's a coincidence that your current 3137 points here are an anagram of 1337 :-) – Stefan Pochmann Jan 03 '16 at 03:49
  • 1
    Turning my research in a different direction, it seems like the answer (either way) seems like it lies in the [mutual recursion article](https://en.wikipedia.org/wiki/Mutual_recursion#Basic_examples). The bit you reference seems to be valid for tail calls using mutual recursion. Though I'm unsure if it qualifies for constant stack space. – Nuclearman Jan 03 '16 at 14:31
  • 1
    @StefanPochmann I updated the Wikipedia article to say `O(log n)` auxiliary space since Shawn Ligocki said it's fine to change it. If you add an answer to this question that says, "Median-of-medians does require `O(log n)` space," I'll accept it. – John Kurlak Jan 04 '16 at 17:26

2 Answers2

12

While I can't rule out that O(1) is possible, that Wikipedia information appears to be a mistake.

  • The implementation shown there takes O(log n) and not just O(1).
  • It's absolutely not obvious how to implement it with O(1) and there's no explanation/reference for it.
  • I asked the author who originally added that information and he replied that he doesn't remember and that it's probably a mistake.
  • A paper from 2005 devoted to solving the selection problem with O(n) time and O(1) extra space says BFPRT (aka Median of Medians) uses Θ(log n) extra space. On the other hand, the paper's main result is that O(n) time and O(1) extra space is possible, and one of the two algorithms presented as proof is some "emulation" of BFPRT. So in that sense it's possible, but I think the emulation rather makes it a different algorithm and the O(1) shouldn't be attributed to "regular" BFPRT. At least not without explanation.
    (Thanks to Yu-HanLyu for showing that paper and more in the comments)
Stefan Pochmann
  • 24,379
  • 7
  • 36
  • 92
  • 1
    The last point assumes that the input is read-only. If the input is modifiable, median finding can be done in place. – Yu-Han Lyu Jan 05 '16 at 12:42
  • @Yu-HanLyu Are you sure? The paper's introduction says that there are two types of constant-working-space algorithms, one being allowed to modify the input. And it lists in-place heapsort and quicksort as examples. That said, I guess we can indeed make the median-of-medians algorithm take only O(1) extra space if we rule out a few of the input numbers, replace them with numbers from the end, and then abuse that end to store recursion information. Is that what you're thinking of as well, or is there a way that at least only rearranges the input without losing data? – Stefan Pochmann Jan 05 '16 at 13:39
  • 1
    I don't know whether median-of-medians algorithm can be done in-place. However, I believe that there are some in-place selection algorithms. See http://link.springer.com/chapter/10.1007%2FBFb0015429 http://www.cai.sk/ojs/index.php/cai/article/viewArticle/345 and http://link.springer.com/chapter/10.1007%2F3-540-19487-8_2 I guess the last one is close to what you think and this paper also mentioned that median-of-medians algorithm uses uses O(log n) extra space. – Yu-Han Lyu Jan 05 '16 at 20:47
  • But the in-place version may be even slower than the original version like the stackless version of quicksort is slower than the original quicksort. – Yu-Han Lyu Jan 05 '16 at 21:02
  • @Yu-HanLyu Sorry, wanted to look a bit more into those papers first. But I removed that last point now and will add a replacement tomorrow (too tired to formulate it now). – Stefan Pochmann Jan 11 '16 at 04:41
  • @Yu-HanLyu Oh well, wrote something now. Does it sound alright? Thanks a lot, btw. – Stefan Pochmann Jan 11 '16 at 05:11
  • @StefanPochmann I guess it is better to clarify the problem. 1. Is original BFPRT can be done in O(1) space? no, since BFPRT takes Θ(log n) space. 2. Is it possible to modify BFPRT so that the algorithm only requires O(1) space? Yes, assuming inputs are modifiable and each variable takes O(1) space. If you consider bit complexity, the algorithm still needs O(lg n) bits. 3. Is it possible to solve selection problem with O(1) space when inputs are read-only? Still an open question. Does it seem right? – Yu-Han Lyu Jan 11 '16 at 11:45
  • To be precise, the last question I raised should be "is it possible to solve selection problem in linear time with O(1) variables when inputs are read-only?" Someone may also ask whether selection problem can be solved in liner time with O(1) variables when inputs are modifiable, so that when algorithm terminates, the order should be identical as the input. That is, algorithms are allowed to swap elements temporarily, but has some mechanisms to restore the original input like Morris traversal. – Yu-Han Lyu Jan 11 '16 at 12:14
  • @Yu-HanLyu Yes, I think that's all right. I'd say read-only input or restoring the input are quite a bit out of scope of the question, though, so I remain happy with my current answer. If you feel that more information should be given, I suggest you write your own answer. I'd probably upvote it :-) – Stefan Pochmann Jan 11 '16 at 15:48
-2

It is O(1).

Let us say we start with an array of length n, and we intend to find the kth element of the sorted list.

After the first call median of medians will spit out a smaller array, now we need to evaluate the ith element of this smaller array. Note that the ith element of this smaller array is the result, so i do not need to pass back any information to the earlier call.

In quick sort, i need to put back the sorted small arrays back into the correct position and so recursion occurs. With median of medians, after the loop (tail recursion), i will be left with the answer.

Recursion depth = O(1)

kudalf
  • 1
  • 1
    Finding the pivot to partition the array is what takes O(lg n) memory. You cannot split out a smaller array until you 1) find the pivot and 2) partition the array around that pivot. You're right that once you've partitioned the array, you don't need to pass any information back to the original caller, but you're ignoring the fact that finding the pivot takes O(lg n) memory. – John Kurlak Feb 27 '17 at 05:01