-1

In Short

I create an BatchBlock which links to an ActionBlock, which calls a async method, but this method never fires.

Details

Here is how I create the blocks

        var instance = new ConnectionSaveStep(repo, progress, total);
        var batch = new BatchBlock<LandingPageConnection>(10000);
        saveAction = new ActionBlock<IEnumerable<LandingPageConnection>>(i => instance.Save(i));
        batch.LinkTo(saveAction);

The save method looks like this:

    internal async Task Save(IEnumerable<LandingPageConnection> pages)
    {
        Trace.WriteLine("Inserting " + pages.Count() + " items ...");
        await repo.InsertBatchAsync(pages);
    }

Elements (about 1 mio) are queued within a Parallel.ForEach .

 Parallel.ForEach(cities, city=>
          { 
             var pages= BuildLandingPage(city);
             batch.Post(pages);

but the save Method gets never raised

When I've a look at the property of the batch and saveAction I can see that first the number of the items in the unput queue of the batch block is increasing till it reaches the batchblock size. After that one item with 10.000 work items is queued in the saveACtion input queue .

But the save is never called.

What am I doing wrong?

Boas Enkler
  • 11,465
  • 12
  • 61
  • 128
  • 1
    Does it work if you don't use `Parallel.ForEach`? – i3arnon Feb 14 '16 at 19:24
  • good Idea i'll give it a shot. But from my expecation this should also work with a parallel for each – Boas Enkler Feb 14 '16 at 19:25
  • 3
    There's nothing in your code that seems problematic. But maybe the `Parallel.ForEach` occupies the `ThreadPool` completely and doesn't leave room for the `ActionBlock`. – i3arnon Feb 14 '16 at 19:27
  • hmm yes maybe so I 'll also try to limit the num,ber of threads – Boas Enkler Feb 14 '16 at 19:28
  • The contents of ParallelForEach should be a TransformBlock themselves, translating cities to pages, with a MaxDOP>1. If `BuildLandingPage` takes long enough to deserv eParallel.ForEach, it also deserves its own block. Also change `Post` to `await SendAsync` to delay posting if the target block gets flooded. `Post` returns a boolean that you have to check while SendAsync allows you to await until the block is able to receive the next message – Panagiotis Kanavos Feb 15 '16 at 10:03
  • 2
    You are right but primarly i want to under stand why it doesn't work. The reason was there was a small code snippet below the code i posted which raised a nullrefence exception – Boas Enkler Feb 15 '16 at 10:23

1 Answers1

0

The solution is in the comments. The reason was that the producer created to many threads, and thats why the consumer never had the chance to fire.

Boas Enkler
  • 11,465
  • 12
  • 61
  • 128