1

I have created these two actions:

public string Sync()
{
    Thread.Sleep(2000);
    return "Hello, Sync!";
}

public async Task<string> Async()
{
    await Task.Delay(2000);
    return "Hello, Async!";
}

and I hosted them under IIS (Windows 10). I created load tests with Visual Studio 2015 Ultimate with local environment as a laboratory. I'm testing with constant pattern with 100 virtual users.

My expectations are that the async action will have better results (more tests run and better average page response time) when number of threads for application pool is less then 100. But I get the same results for both tests. Both tests are set to run for 1 minutes.

I tried modifying plenty of options. In aspnet.config I set the following settings:

I tried setting to 8, 12, 100, always same results for both sync and async methods:

17 seconds average page time Total tests: 289 4.82 Tests/Sec

Any ideas what I am doing wrong? I'm expecting the async method to have 2 seconds average page time, since no threads should be blocked.

Gorgi Rankovski
  • 2,223
  • 1
  • 20
  • 30
  • The async version shouldn't complete first. It should require less resources. – i3arnon Dec 16 '15 at 23:34
  • i3arnon I'm trying to set the max number of threads for the application to be low. If it is set to 12, and I run 13 requests to the sync method, then one request will have to wait 2 seconds before it starts executing. If I run 13 requests to the async, they should complete all withing the same time. – Gorgi Rankovski Dec 16 '15 at 23:38
  • 1
    Note that there are default limitations to making concurrent Http calls in a .NET client. I would investigate your test client to make sure it is in fact firing hundreds of concurrent requests or just a few. – Bringer128 Dec 16 '15 at 23:44

2 Answers2

2

It turns out, IIS for non-Windows Server editions has limit of max 10 simultaneous requests: Does IIS 7 have limit of simultaneous requests?

I wish I knew this earlier... But, I hope it helps someone in the future.

Community
  • 1
  • 1
Gorgi Rankovski
  • 2,223
  • 1
  • 20
  • 30
1

I tried modifying plenty of options. In aspnet.config I set the following settings:

Sounds like you tried to cripple ASP.NET so that the async version comes out ahead.

As long as threads are not scarce async IO is not at an advantage. It uses less threads and less memory while it is running but has very slightly higher CPU usage. None of this matters as long as we are not in the 100s of threads.

If you want to see this in action run 1000 concurrent requests. You will see a difference. I'm not sure what "virtual users" are and ASP.NET does not care. What matters is the effective degree of parallelism/concurrency.

You can contrive arbitrarily high or low differences. The average number of threads needed is requestsPerSecond * requestDurationInSeconds.

I'll link you my standard posts on whether to go sync or async because I feel you don't have absolute clarity for when async IO is appropriate.

https://stackoverflow.com/a/25087273/122718 Why does the EF 6 tutorial use asychronous calls? https://stackoverflow.com/a/12796711/122718 Should we switch to use async I/O by default?

Community
  • 1
  • 1
usr
  • 162,013
  • 33
  • 219
  • 345
  • Thanks for the answer usr. I'm using Visual Studio Load Testing: https://msdn.microsoft.com/library/dd293540(v=vs.110).aspx and virtual users is a term from there. Basically its the number of requests executed simultaneously. .NET's async/await is not strictly bound to async IO, but to system events. Task.Delay doesn't waste a thread or the CPU, but its bound to Windows timers. So, its a perfect match to simulate async IO. It worked for Scott: http://www.hanselman.com/blog/InstallingAndRunningNodejsApplicationsWithinIISOnWindowsAreYouMad.aspx – Gorgi Rankovski Dec 16 '15 at 23:47
  • Yeah, Delay is IO in my mind. It's certainly not CPU. Does this answer the question? I can't quite tell if you disagree with something. – usr Dec 16 '15 at 23:58
  • My theory is that the Sync method will hang the thread for 2 seconds. If I set the number of threads to 12 and I execute 13 requests against it, 12 request will finish in 2 seconds, while the 13th request in 4 seconds. If I do the same with the async method, all 13 requests should finish in 2 seconds. I can't make this happen, so I'm not sure whether my problem is in IIS or in Visual Studio load test settings... – Gorgi Rankovski Dec 17 '15 at 00:03
  • 1
    Theory correct. Start 1000 threads. If it does not work the problem is the client. Also note that client Windows limits to 10 concurrent requests. – usr Dec 17 '15 at 00:08
  • Unless these tests are either; over a network to a Windows Server or run on a Windows Server, the results will not be real-world. Additional outgoing requests are also throttled by .Net, and you'd have to increase it's default maximum outgoing request limit. – Erik Philips Dec 17 '15 at 00:42
  • @ErikPhilips because of the long wait time of 2sec the network is not a bottleneck. It will be nearly idle. Doesn't make a difference. Regarding outgoing requests, there are none since he is using some Visual Studio load testing framework to generate them. – usr Dec 17 '15 at 00:50
  • @usr you mis-understand, I was saying that in both scenario's of testing, it should be against Windows Server. **If** over a network, then .Net will be a bottleneck on the out-going requests. – Erik Philips Dec 17 '15 at 01:01
  • Eric, you're right - test should be pointed to Windows Server. I was using Windows 10, which has limits. See my answer. – Gorgi Rankovski Dec 17 '15 at 07:40