How to create a non-blocking @RestController webservice in spring?

Question

I'm having a @RestController webservice method that might block the response thread with a long running service call. As follows:

@RestController
public class MyRestController {
    //could be another webservice api call, a long running database query, whatever
    @Autowired
    private SomeSlowService service;

    @GetMapping()
    public Response get() {
        return service.slow();
    }

    @PostMapping()
    public Response get() {
        return service.slow();
    }
}

Problem: what if X users are calling my service here? The executing threads will all block until the response is returned. Thus eating up "max-connections", max threads etc.

I remember some time ago a read an article on how to solve this issue, by parking threads somehow until the slow service response is received. So that those threads won't block eg the tomcat max connection/pool.

But I cannot find it anymore. Maybe somebody knows how to solve this?

You *may* want to work with sockets. Client informs server -> Server does stuff (immediatly returns 200) -> Server informs client via socket. E.g. if the client is written in Javascript you may want to use *socket.js* — Lino, May 09 '19 at 08:26
maybe ``@Async`` from spring in the service. Then return 202 Accepted with an uri to follow progression — spi, May 09 '19 at 08:26
@spi: no, that would of course give me `Future` for the service calll itself. But *still* the *http thread* will be in blocking state, eating up max tomcat connections for example, and that's the target here. — membersound, May 09 '19 at 08:27
@Lino: yes that could be an option. But I remember in "that article" there was a programmatic solution to this, without having to force the client to request the response itself async. It's just the question of: how can I **park** those webservice threads so they don't block until I write the response out. — membersound, May 09 '19 at 08:29
https://docs.spring.io/spring/docs/current/spring-framework-reference/web.html#mvc-ann-async — JB Nizet, May 09 '19 at 08:44
@JBNizet is there any (dis)advantage of `DeferredResult` over `Callable`? Should one of them be preferred over eg `CompletableFuture`? — membersound, May 09 '19 at 09:35
I can't say more than what the documentation says. I've never used them. — JB Nizet, May 09 '19 at 09:36

g00glen00b · Answer 1 · 2019-05-10T10:55:13.310

there are a few solutions, such as working with asynchronous requests. In those cases, a thread will become free again as soon as the CompletableFuture, DeferredResult, Callable, ... is returned (and not necessarily completed).

For example, let's say we configure Tomcat like this:

server.tomcat.max-threads=5 # Default = 200

And we have the following controller:

@GetMapping("/bar")
public CompletableFuture<String> getSlowBar() {
    return CompletableFuture.supplyAsync(() -> {
        silentSleep(10000L);
        return "Bar";
    });
}

@GetMapping("/baz")
public String getSlowBaz() {
    logger.info("Baz");
    silentSleep(10000L);
    return "Baz";
}

If we would fire 100 requests at once, you would have to wait at least 200 seconds before all the getSlowBar() calls are handled, since only 5 can be handled at a given time. With the asynchronous request on the other hand, you would have to wait at least 10 seconds, because all requests will likely be handled at once, and then the thread is available for others to use.

Is there a difference between CompletableFuture, Callable and DeferredResult? There isn't any difference result-wise, they all behave the similarly.

The way you have to handle threading is a bit different though:

With Callable, you rely on Spring executing the Callable using a TaskExecutor
With DeferredResult you have to to he thread-handling by yourself. For example by executing the logic within the ForkJoinPool.commonPool().
With CompletableFuture, you can either rely on the default thread pool (ForkJoinPool.commonPool()) or you can specify your own thread pool.

Other than that, CompletableFuture and Callable are part of the Java specification, while DeferredResult is a part of the Spring framework.

Be aware though, even though threads are released, connections are still kept open to the client. This means that with both approaches, the maximum amount of requests that can be handled at once is limited by 10000, and can be configured with:

server.tomcat.max-connections=100 # Default = 10000

Thanks for the detailed insight! Is there any advantage for using one of the async classes? Or is it just a matter of taste? — membersound, May 09 '19 at 10:16
@membersound As I mentioned in my answer, they differ in how much control you have over the thread pools that are being used. Additionally, since `Callable` and `CompletableFuture` are part of the Java spec, they come with a richer eco-system. For example, you can find libraries like [completable-futures](https://github.com/spotify/completable-futures) that allow you to do more with `CompletableFuture`. If those things matter to you, then it isn't just a matter of taste. — g00glen00b, May 09 '19 at 10:24

score 0 · Answer 2 · answered May 10 '19 at 08:51

in my opinion.the async may be better for the sever.for this particular api, async not works well.the clients also hold the connections. finally it will eating up "max-connections".you can send the request to messagequeue(kafka)and return success to clients. then you get the request and pass it to the slow sevice.

How to create a non-blocking @RestController webservice in spring?

2 Answers2

Linked