LINQ SelectMany and Where extension method ignoring nulls

Question

I have the below example code, and I am interested to know how I can make this any cleaner, possibly through better use of SelectMany(). At this point the QuestionList property will not be null. All I want is a list of answerRows that are not null, but Questions can sometimes be null too.

IEnumerable<IQuestion> questions = survey.QuestionList
                    .Where(q => q.Questions != null)
                    .SelectMany(q => q.Questions);
            
if(questions == null)
return null;

IEnumerable<IAnswerRow> answerRows = questions
                    .Where(q => q.AnswerRows != null)
                    .SelectMany(q => q.AnswerRows);

if(answerRows == null)
return null;

I was interested by Jon's comment about Enumerable.SelectMany and Null.. so I wanted to try my example with some fake data to more easily see where the error is, please see the below, specifically how I am using SelectMany() on the result of a SelectMany(), its clearer to me now that the problem was having to make sure you don't use SelectMany() on a null reference, obvious when I actually read the NullReferenceException name :( and finally put things together.

Also while doing this, I realised that the use of try { } catch() { } in this example is useless and as usual Jon Skeet has the answer :) deferred execution..

so if you want to see the exception for row 2, comment out the relevant row 1 bits :P, sorry I couldn't figure out how to stop this error without re-writing the code example.

using System;
using System.Collections.Generic;
using System.Linq;

namespace SelectManyExample
{
    class Program
    {
        static void Main(string[] args)
        {
            var questionGroupList1 = new List<QuestionGroup>() {
                new QuestionGroup() {
                    Questions = new List<Question>() {
                        new Question() {
                            AnswerRows = new List<AnswerRow>() {
                                new AnswerRow(),
                                new AnswerRow()
                            }
                        },

                        // empty question, causes cascading SelectMany to throw a NullReferenceException
                        null,

                        new Question() {
                            AnswerRows = new List<AnswerRow>() {
                                new AnswerRow() {
                                    Answers = new List<Answer>() {
                                        new Answer(),
                                        new Answer()
                                    }
                                }
                            }
                        }
                    }
                }
            };

            var questionGroupList2 = new List<QuestionGroup>() {
                null,
                new QuestionGroup()
            };

            IEnumerable<AnswerRow> answerRows1 = null;
            IEnumerable<AnswerRow> answerRows2 = null;

            try
            {
                answerRows1 = questionGroupList1
                    .SelectMany(q => q.Questions)
                    .SelectMany(q => q.AnswerRows);
            }
            catch(Exception e) {
                Console.WriteLine("row 1 error = " + e.Message);
            }

            try
            {
                answerRows2 = questionGroupList2
                    .SelectMany(q => q.Questions)
                    .SelectMany(q => q.AnswerRows);
            }
            catch (Exception e)
            {
                Console.WriteLine("row 2 error = " + e.Message);
            }


            Console.WriteLine("row 1: " + answerRows1.Count());
            Console.WriteLine("row 2: " + answerRows2.Count());
            Console.ReadLine();
        }


    }

    public class QuestionGroup {
        public IEnumerable<Question> Questions { get; set; }
    }

    public class Question {
        public IEnumerable<AnswerRow> AnswerRows { get; set; }
    }

    public class AnswerRow {
        public IEnumerable<Answer> Answers { get; set; }
    }

    public class Answer {
        public string Name { get; set; }
    }
}

`questions` and `answerRows` can never be `null`. And in a sane design, `q.Questions` and `q.AnswerRows` probably should never be `null` as well. — Jon, Jan 22 '13 at 22:25
@Jon Explain why they can never be null. If `QuestionList` is `List` of length 1, there's no reason I can think of stopping `QuestionList[0]` from being null. Similarly, 'Type' in the my example can still have property 'Questions' uninitialized. i.e. `null`. — Paul Fleming, Jan 22 '13 at 22:31
@flem They can never be null because that's how `Where` and `SelectMany` work. If the input is null they throw an exception, and if it's not null the result will either be a sequence of items, or an empty sequence. It will *never* be `null`. As a rule you should avoid `null` values for collections or sequences, just use an empty collection instead. — Servy, Jan 22 '13 at 22:32
It would break if `q` is null, not if `q.Questions` is null. — Paul Fleming, Jan 22 '13 at 22:33
@flem: That. Also `questions[0]` does not compile because `questions` is not an `IList`. And even if it did, that's not the same as `questions`. — Jon, Jan 22 '13 at 22:34
In my case the problem is with posted form values and model binding to this existing structure, in one case the model binder ignores a question entirely when its answers are not filled in, a specific check box list case... I would much rather they were always not null.. but regardless of the example I wanted to know more about how to use .Where and .SelectMany together. — Pricey, Jan 22 '13 at 22:34
@Jon Am I missing an assumption here. This could be linq to objects, entities, xml, sql. That's not indicated as far as I can see. Where is this definition of `questions`? As far as I can see, it could be anything as long as it implements `IEnumerable`. — Paul Fleming, Jan 22 '13 at 22:36
@flem: `questions` is whatever `Enumerable.SelectMany` or `Queryable.SelectMany` decides to return. And that's guaranteed to be non-null. — Jon, Jan 22 '13 at 22:37
@Jon. That doesn't apply on the `.Where`. `q.Questions` can be any `IEnumerable` including `IList`. Besides `q.Questions[0]` was just to explain my `null` possibility example. — Paul Fleming, Jan 22 '13 at 22:39
@flem `q.Questions` is entirely different from `questions`. C# is case sensitive. `questions` is a local variable that will *never* be null. `q.Questions` exists in two contexts, since `q` is re-used as a local variable in two scopes. In both cases it could be null, but as Jon first said, they really *shouldn't* be null (by convention). — Servy, Jan 22 '13 at 22:41
@flem: Not to mention that of course it also applies to the `Where`. No LINQ method will *ever* return `null`. — Jon, Jan 22 '13 at 22:42
@Servy. I think we're on the same page. I've never been referring to the variable `questions`, only the property `Questions` on parameter `q` in the expression in the initial `.Where`. ;) — Paul Fleming, Jan 22 '13 at 22:43
@Jon. As just said, I was never referring to the queries, but rather the expressions. — Paul Fleming, Jan 22 '13 at 22:44
@flem Well, Jon was, he made that clear, and you were referring to him, so yes, you were referring to the variable `questions`. There isn't even any other identifier with the same name. — Servy, Jan 22 '13 at 22:44

score 58 · Accepted Answer · answered Jan 22 '13 at 22:25

survey.QuestionList
    .Where(l => l.Questions != null)
    .SelectMany(l => l.Questions)
    .Where(q => q != null && q.AnswerRows != null)
    .SelectMany(q => q.AnswerRows);

I'd recommend you ensure your collections are never null. null can be a bit of a nuisance if you don't handle it well. You end up with if (something != null) {} all over your code. Then use:

survey.QuestionList
    .SelectMany(l => l.Questions)
    .SelectMany(q => q.AnswerRows);

Servy · Answer 2 · 2013-01-22T22:45:53.473

11

public static IEnumerable<TResult> SelectNotNull<TSource, TResult>(
    this IEnumerable<TSource> source, Func<TSource, IEnumerable<TResult>> selector)
    where TResult : class
{
    return source.Select(selector)
        .Where(sequence => sequence != null)
        .SelectMany(x => x)
        .Where(item => item != null);
}

This then allows you to do the following:

var allAnswers = survey.QuestionList
    .SelectNotNull(list => list.Questions)
    .SelectNotNull(question => question.AnswerRows);

edited Jan 22 '13 at 22:45

answered Jan 22 '13 at 22:38

Servy

193,745
23
295
406

1

"where TResult : class" — is unnecessary limitation preventing usage of long and int sequences. Last "where" is also not needed, if necessary one can add it 'outside'. – greatvovan Aug 08 '16 at 21:27
@greatvovan "one can add it outside" is not true because the extension method returns an IEnumerable which has no possibilities to add (or remove)... – Bernoulli IT Apr 18 '17 at 13:02
3

@BernoulliIT greatvovan was referring to adding a call to *the extension method* to the end of a query on the calling side, not to adding items to sequences. – Servy Apr 18 '17 at 13:13
SelectManyNotNull is more appropriate name, than SelectNotNull – Michael Freidgeim Sep 22 '20 at 13:12

score 5 · Answer 3 · answered Jul 01 '19 at 08:36

A solution that complies with DRY would be to use the null-coalescing operator ?? in your SelectMany lambda expression.

IEnumerable<IQuestion> questions = survey.QuestionList.SelectMany(q => q.Questions ?? Enumerable.Empty<IQuestion>());

IEnumerable<IAnswerRow> answerRows = questions.SelectMany(q => q.AnswerRows ?? Enumerable.Empty<IAnswerRow>());

In both the OP's code and the above code, questions and answerRows will never be null, so the null checks are not required (you may wish to put .Any() checks depending on your business logic). But the above code will also never result in an exception if q.Questions or q.AnswerRows is null.

LINQ SelectMany and Where extension method ignoring nulls

3 Answers3