LINQ Lambda vs Query Syntax Performance

Question

I saw a LINQ query syntax in my project today which was counting items with a specific condition from a List like this:

int temp = (from A in pTasks 
            where A.StatusID == (int)BusinessRule.TaskStatus.Pending     
            select A).ToList().Count();

I thought of refactoring it by rewriting it using Count(Func) to make it more readable. I thought it would also be good performance-wise, so I wrote:

int UnassignedCount = pTasks.Count(x => x.StatusID == (int)BusinessRule.TaskStatus.Pending);

But when I check using StopWatch, the time elapsed by the lambda expression is always more than the query syntax:

Stopwatch s = new Stopwatch();
s.Start();
int UnassignedCount = pTasks.Count(x => x.StatusID == (int)BusinessRule.TaskStatus.Pending);
s.Stop();
Stopwatch s2 = new Stopwatch();
s2.Start();
int temp = (from A in pTasks 
            where A.StatusID == (int)BusinessRule.TaskStatus.Pending
            select A).ToList().Count();
s2.Stop();

Can somebody explain why it is so?

Answer 1

I have simulated your situation. And yes, there is difference between execution times of these queries. But, the reason of this difference isn't syntax of the query. It doesn't matter if you have used method or query syntax. Both yields the same result because query expres­sions are trans­lated into their lambda expres­sions before they’re com­piled.

But, if you have paid attention the two queries aren't same at all.Your second query will be translated to it's lambda syntax before it's compiled (You can remove ToList() from query, because it is redundant):

pTasks.Where(x => x.StatusID == (int)BusinessRule.TaskStatus.Pending).Count();

And now we have two Linq queries in lambda syntax. The one I have stated above and this:

pTasks.Count(x => x.StatusID == (int)BusinessRule.TaskStatus.Pending);

Now, the question is:
Why there is difference between execution times of these two queries?

Let's find the answer:
We can understand the reason of this difference by reviewing these:
- .Where(this IEnumerable<TSource> source, Func<TSource, bool> predicate).Count(this IEnumerable<TSource> source)
and
- Count(this IEnumerable<TSource> source, Func<TSource, bool> predicate);

Here is the implementation of Count(this IEnumerable<TSource> source, Func<TSource, bool> predicate):

public static int Count<TSource>(this IEnumerable<TSource> source, Func<TSource, bool> predicate)
{
    if (source == null) throw Error.ArgumentNull("source");
    if (predicate == null) throw Error.ArgumentNull("predicate");
    int count = 0;
    foreach (TSource element in source) {
        checked {
            if (predicate(element)) count++;
        }
    }
    return count;
}

And here is the Where(this IEnumerable<TSource> source, Func<TSource, bool> predicate):

public static IEnumerable<TSource> Where<TSource>(this IEnumerable<TSource> source, Func<TSource, bool> predicate)
{
    if (source == null) 
        throw Error.ArgumentNull("source");
    if (predicate == null) 
        throw Error.ArgumentNull("predicate");
    if (source is Iterator<TSource>) 
        return ((Iterator<TSource>)source).Where(predicate);
    if (source is TSource[]) 
        return new WhereArrayIterator<TSource>((TSource[])source, predicate);
    if (source is List<TSource>) 
        return new WhereListIterator<TSource>((List<TSource>)source, predicate);
    return new WhereEnumerableIterator<TSource>(source, predicate);
}

Let's pay an attention to Where() implementation. It will return WhereListIterator() if your collection is List, but Count() will just iterate over source. And in my opinion they have made some speed up in the implementation of WhereListIterator. And after this we are calling Count() method which takes no predicate as input and only will iterate on filtered collection.

---

And regarding to that speed up in the implementation of WhereListIterator:

I have found this question in SO: LINQ performance Count vs Where and Count. You can read @Matthew Watson answer there. He explains the performance difference between these two queries. And the result is: The Where iterator avoids indirect virtual table call, but calls iterator methods directly. As you see in that answer call instruction will be emitted instead of callvirt. And, callvirt is slower than call:

From bookCLR via C#:

When the callvirt IL instruction is used to call a virtual instance method, the CLR discovers the actual type of the object being used to make the call and then calls the method polymorphically. In order to determine the type, the variable being used to make the call must not be null. In other words, when compiling this call, the JIT compiler generates code that verifes that the variable’s value is not null. If it is null, the callvirt instruction causes the CLR to throw a NullReferenceException. This additional check means that the callvirt IL instruction executes slightly more slowly than the call instruction.

Answer 2

Like Farhad said, the implementation of Where(x).Count() and Count(x) vary. The first one instantiates an additional iterator, which on my pc costs about 30.000 ticks (regardless of the collection size)

Also, ToList is not free. It allocates memory. It costs time. On my pc, it roughly doubles execution time. (so linear dependent op the collection size)

Also, debugging requires spin-up time. So it's difficult to accurately measure performance in one go. I'd recommend a loop like this example. Then, ignore the first set of results.

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace ConsoleApplication1
{
    class Program
    {
        static void Main(string[] args)
        {
            var pTasks = Task.GetTasks();
            for (int i = 0; i < 5; i++)
            {

                var s1 = Stopwatch.StartNew();
                var count1 = pTasks.Count(x => x.StatusID == (int) BusinessRule.TaskStatus.Pending);
                s1.Stop();
                Console.WriteLine(s1.ElapsedTicks);

                var s2 = Stopwatch.StartNew();
                var count2 =
                    (
                        from A in pTasks
                        where A.StatusID == (int) BusinessRule.TaskStatus.Pending
                        select A
                        ).ToList().Count();
                s2.Stop();
                Console.WriteLine(s2.ElapsedTicks);

                var s3 = Stopwatch.StartNew();
                var count3 = pTasks.Where(x => x.StatusID == (int) BusinessRule.TaskStatus.Pending).Count();
                s3.Stop();
                Console.WriteLine(s3.ElapsedTicks);


                var s4 = Stopwatch.StartNew();
                var count4 =
                    (
                        from A in pTasks
                        where A.StatusID == (int) BusinessRule.TaskStatus.Pending
                        select A
                        ).Count();
                s4.Stop();
                Console.WriteLine(s4.ElapsedTicks);

                var s5 = Stopwatch.StartNew();
                var count5 = pTasks.Count(x => x.StatusID == (int) BusinessRule.TaskStatus.Pending);
                s5.Stop();
                Console.WriteLine(s5.ElapsedTicks);
                Console.WriteLine();
            }
            Console.ReadLine();
        }
    }

    public class Task
    {
        public static IEnumerable<Task> GetTasks()
        {
            for (int i = 0; i < 10000000; i++)
            {
                yield return new Task { StatusID = i % 3 };
            }
        }

        public int StatusID { get; set; }
    }

    public class BusinessRule
    {
        public enum TaskStatus
        {
            Pending,
            Other
        }
    }
}