While we drop the constant in big O notation, does it matter in real-life situations?

Question

While I understand that big O notation simply describes the growth rate of an algorithm, I'm uncertain if there is any difference in efficiency in real life between the following O(n) algorithms.

To print the value of a node in a linked list k places from the end of the list.

Given a node:

/* Link list node */
struct node
{
  int data;
  struct node* next;
};

Solution 1 O(n)

This solution iterates over the list twice, once to find the length of the list, and the second time to get to the end of the list - N.

void printNthFromLast(struct node* head, int n)
{
    int len = 0, i;
    struct node *temp = head;


    // 1) Count the number of nodes in Linked List
    while (temp != NULL)
    {
        temp = temp->next;
        len++;
    }

    // Check if value of n is not more than length of the linked list
    if (len < n)
      return;

    temp = head;

    // 2) Get the (n-len+1)th node from the begining
    for (i = 1; i < len-n+1; i++)
    {
       temp = temp->next;
    }
    printf ("%d", temp->data);

    return;
}

Solution 2 O(n)

This solution only iterates over the list once. The ref_ptr pointer leads, and a second pointer (main_ptr) follows it k places behind. When ref_ptr reaches the end of the list, main_ptr should be pointing at the correct node (k places from the end of the list).

void printNthFromLast(struct node *head, int n)
{
  struct node *main_ptr = head;
  struct node *ref_ptr = head;

  int count = 0;
  if(head != NULL)
  {
    while( count < n )
     {
        if(ref_ptr == NULL)
        {
           return;
        }
        ref_ptr = ref_ptr->next;
        count++;
     }

     while(ref_ptr != NULL)
     {
        main_ptr = main_ptr->next;
        ref_ptr  = ref_ptr->next;
     }
  }
}

The question is: Even though both solutions are O(n) while leaving big O notation aside, is the second solution more efficient that the first for a very long list as it only iterates over the list once?

Answer 1

Yes. In the specific example where the same work occurs, a single loop is likely to be more efficient than looping over a set of data twice. But the idea of O(2n) ~ O(n) is that 2 ns vs 1 ns may not really matter. Big O works better to show how a piece of code might scale, e.g. if you made the loop O(n^2) then the difference of O(n) vs O(2n) is much less than O(n) vs O(n^2).

If your linked list contains terrabytes of data, then it might be worth reducing to the single loop iteration. A big O metric, in this case may not be sufficient to describe your worst case; you would be better off timing the code and considering the needs of the application.

Another example is in embedded software, where 1 ms vs 2 ms could be the difference between a 500 Hz and a 1 kHz control loop.

The lesson learned is that it depends on the application.

Answer 2

The constant only matters if the order is the same, and the operations are comparable in complexity. If they aren't of the same order, then the one with the higher order is guaranteed to take longer once you have a large enough n. Sometimes n must be larger than your typical data set, and the only way to pick the most efficient algorithm is to benchmark them.

Answer 3

I think from my point of view the difference between two routines which are O(n) and O(n), for example, is not really the point of the O notation. The key differences are between O(n^2) and O(n), for example. [n^2 is of course n squared]

So in general the power, p, for O(n^p) is critical in how the efficiency of a routine scales with size.

So looking at the two routines you have there may be differences in performance between them, but to a first approximation they will behave similarly as the dataset size increases.

An example of code where scaling is key is the Fourier Transform where some methods give O(n^2) and others give O(n log n).

Answer 4

While in your particular example, it is too close to tell since compiler optimizations, caching, data access rates, and many other issues complicate matters, to answer your title question "While we drop the constant in big O notation does it matter in real life situations" is easy:

Yes.

Imagine we have a very time-consuming function F that, for a given input, always produces the same output.

We have a loop that must execute N times. In this loop, we use the return value of F several times in order to calculate something.

The input(s) to F are the always same for a given iteration of this loop.

We have two potential implementations of this loop in mind.

1. Implementation #1:

   loop:
       set inputs to something;
       value = F(inputs);
       do something with value;
       do something else with value;
       do something else else with value;
   done
   

2. Implementation #2:

   loop:
       set inputs to something;
       value = F(inputs);
       do something with value;
       value = F(inputs);
       do something else with value;
       value = F(inputs);
       do something else else with value;
   done
   

Both implementations loop the same number of times. Both get the same result. Obviously implementation #2 is less efficient, since it does more work per iteration.

In this trivial example, the compiler might notice that F always returns the same value for the same input, and it might notice that we call it with the same inputs each time, but for any compiler, we can construct an example that does the equivalent of O(C*n) vs O(n) where C really matters in practice.

Answer 5

Yes, it might make a difference. I didn't check your code for being correct, but consider this:

The first solution loops over the list once up to the end and another time up to n. The second solution loops over the list once, but it uses ->next() on a second pointer n times. So basically they should call ->next() about the same amount of times (maybe +-1 or so).

Independent from your example, that's not what big O notation is about. It is about giving an approximation to how an algorithm scales if the amount of data increases. If you have an algorithm O(n) and reduce its runtime by 10% (independent of how you do that) then of course it's a benefit. But if you double the data, its runtime will still double, and that's what O(n) notation is about. (An O(n^2) algorithm e.g. will have its runtime scaled by a factor of 4 if you double the data.)

Answer 6

This is the kind of question people ask when they are transitioning from academia to practicality.

Certainly big-O matters if your data sets are likely to be very large, where "very large" is for you to decide. Sometimes, data set size is a primary concern. Certainly not always.

Regardless, large data or not, there are always constant factors, and they can make the difference between seconds and hours. You definitely care about them.

What is generally not taught in school is how to find big speedup factors. For example, in perfectly well-written software big speedups can lurk, as in this example.

The key to getting the speedups is not to miss any. Just finding some, but not all, is not good enough, and most tools have huge blind spots. That link points you to a method that experienced programmers have learned.

Answer 7

The constant certainly matters, and in many cases one may be inclined to say "it's the only thing that matters".

A lot of situations and problems nowadays involve something that has an extraordinarily long latency: cache misses, page faults, disk reads, GPU stalls, DMA transfers. In comparison with these, sometimes it does not matter at all whether you have to do a few thousand or a few ten thousand iterations extra.

ALU power has consistently gone up a lot steeper than memory bandwidth (and, more importantly, latency), or access to other devices such as disks during the last two decades. On GPUs, this is even more pronounced than on CPUs (by the time DMA and ROP get 2-3 times faster, ALU has become 15-20 times faster)

An algorithm with O(log N) complexity (say, a binary search) which causes a single page fault may easily be several thousand times slower than a O(N) algorithm (say, a linear search) that avoids this fault.

Hash tables are O(1) but have repeatedly been shown to be slower than other algorithms with higher complexity. Linked lists generally have the same (or better) algorithmic complexity compared to vectors. However, a vector will almost always significantly outperform a list, due to lists doing more allocations and having more cache misses. Unless objects are huge, even having to move around a few thousand elements in a vector to insert something in the middle is usually faster than a single node allocation and insertion into a list.

Cuckoo hashing was famous for a short time a decade ago because it is O(1) with a guaranteed maximum on the worst case (accessing 2 items). Turned out it was much inferior in practice because you had two practically guaranteed cache misses on every access.

Iterating a two-dimensional array one way or the other (rows first / columns first) is exactly identical in complexity, and even in the number of operations. One, however, has a constant that is a thousand times larger, and will run a thousand times slower.