How to use async lambda with SelectMany?

Question

I'm getting the following error when trying to use an async lambda within IEnumerable.SelectMany:

var result = myEnumerable.SelectMany(async (c) => await Functions.GetDataAsync(c.Id));

The type arguments for method 'IEnumerable System.Linq.Enumerable.SelectMany(this IEnumerable, Func>)' cannot be inferred from the usage. Try specifying the type arguments explicitly

Where GetDataAsync is defined as:

public interface IFunctions {
    Task<IEnumerable<DataItem>> GetDataAsync(string itemId);
}

public class Functions : IFunctions {
    public async Task<IEnumerable<DataItem>> GetDataAsync(string itemId) {
        // return await httpCall();
    }
}

I guess because my GetDataAsync method actually returns a Task<IEnumerable<T>>. But why does Select work, surely it should throw the same error?

var result = myEnumerable.Select(async (c) => await Functions.GetDataAsync(c.Id));

Is there any way around this?

@Grundy `Task>`, but I've added the full declaration to the question. Where `T` is different than the type of `myEnumerable` — CodingIntrigue, Nov 03 '15 at 09:40
@HimBromBeere, select return _collection of collection_, but i think OP need simple collection — Grundy, Nov 03 '15 at 09:41
But `GetDataAsnyc` does not return an enumeration but a `Task`. — HimBromBeere, Nov 03 '15 at 09:43
First of all, Your assumption is right. if your method returns Task, await converts it to **T**. So return type is not the problem. But I really did not get why are you using SelectMany? You don't have something like **list of list** ? — ozgur, Nov 03 '15 at 09:51
That's a case of the [XY Problem](http://meta.stackexchange.com/questions/66377/what-is-the-xy-problem), asking about your attempted solution, not the actual problem. *What* are you trying to execute in parallel? Paralled db calls typically slow performance, they don't increase it. Why not PLINQ anyway? Or `await Task.WhenAll()` on the IEnumerable`? No reason to await inside `Select` — Panagiotis Kanavos, Nov 03 '15 at 09:52
@ozgur This is only partly correct. The await "converts" the `Task` to `T`, but since it's an async lambda it returns a `Task` again, much like a normal async function would. The return type is the problem because it is `Task>`. — Dirk, Nov 03 '15 at 09:53
If you want to retrieve multiple objects, just write the proper query for your ORM/data layer that returns data for a list of IDs. It will be *much* faster than multiple individual queries - you pay the network cost only once while selecting over a range of IDs is trivial. — Panagiotis Kanavos, Nov 03 '15 at 09:53
This is a HTTP request using linq-to-objects to format my code nicely, no data layers involved. @AsadSaeeduddin Perhaps my question is a poor reproduction - there is an IoC container in play here injecting `Functions` into my class. `myEnumerable` is simply `IEnumable` which hopefully is irrelevant as I know my business logic is correct. — CodingIntrigue, Nov 03 '15 at 10:02
@RGraham, possibly you can change `Task>` to `IEnumerable>`? — Grundy, Nov 03 '15 at 10:06

SergeyGrudskiy · Answer 1 · 2016-03-08T18:23:58.657

52

This is an extension:

public static async Task<IEnumerable<T1>> SelectManyAsync<T, T1>(this IEnumerable<T> enumeration, Func<T, Task<IEnumerable<T1>>> func)
{
    return (await Task.WhenAll(enumeration.Select(func))).SelectMany(s => s);
}

That allows you to run:

var result = await myEnumerable.SelectManyAsync(c => Functions.GetDataAsync(c.Id));

Explanation: you have a list of tasks, each returns Task<IEnumerable<T>>. So you need to fire them all, then await all, and then squash the result via SelectMany.

edited Mar 08 '16 at 18:23

answered Mar 08 '16 at 18:17

SergeyGrudskiy

651
1
5
5

2

One advantage of having this extracted to a method, rather than having it inline in your code, is that the extension returns a `Task` so this can be awaited at a later place in the code. – Tim Nov 07 '17 at 15:01
10

Be aware that this parallelizes all calls to OP's `GetDataAsync(...)`. Usually that's a good thing, but might not be desired in certain scenarios. – Good Night Nerd Pride Jan 08 '18 at 16:59

score 18 · Accepted Answer · answered Nov 03 '15 at 09:48

async lambda expression cannot be converted to simple Func<TSource, TResult>.

So, select many cannot be used. You can run in synchronized context:

myEnumerable.Select(c => Functions.GetDataAsync(c.Id)).SelectMany(task => task.Result);

or

List<DataItem> result = new List<DataItem>();

foreach (var ele in myEnumerable)
{
    result.AddRange(await Functions.GetDataAsyncDo(ele.Id));
}

You cannot neither use yield return - it is by design. f.e.:

public async Task<IEnuemrable<DataItem>> Do() 
{
    ...
    foreach (var ele in await Functions.GetDataAsyncDo(ele.Id)) 
    {
        yield return ele; // compile time error, async method 
                          // cannot be used with yield return
    }

}

Thanks a lot, I used `.Result` instead of `await` to make the call syncrhronous and all is well — CodingIntrigue, Nov 03 '15 at 10:04
Downvote was probably due to the use of `.Result` which may [cause deadlocks](https://blog.stephencleary.com/2012/07/dont-block-on-async-code.html). — Good Night Nerd Pride, Jan 08 '18 at 16:56

score 6 · Answer 3 · edited Oct 31 '17 at 08:29

6

Select works because it will return an IEnumerable<Task<T>>, which can then be awaited with e.g. Task.WhenAll.

So, an easy workaround to this problem is:

IEnumerable<Task<IEnumerable<T>>> tasks = source.Select(GetNestedEnumerableTask);
IEnumerable<T>[] nestedResults = await Task.WhenAll(tasks);
IEnumerable<T> results = nestedResults.SelectMany(nr => nr);

edited Oct 31 '17 at 08:29

Community

1
1

answered Oct 19 '17 at 07:45

Andreas Ågren

3,292
20
31

How to use async lambda with SelectMany?

3 Answers3

Linked