This project is to demonstrate the feasibility of applying Spring Web, Kotlin coroutines, and Retrofit2 together.
One of common issues that backend engineers face is calling databases or downstream services in parallel. One solution is to migrate to event driven design, but it will need quite some effort. And more practically, you’d like to parallelize these calls to achieve a low latency.
Making parallel calls is not easy and can be error-prone. This is partly due to the container design behind. In container-based solutions like Spring, Java EE, Servlets, web apps are hosted in web containers (EJB containers) and utilise threads spawned (or allocated from thread pool) by the web container. All processing logic, downstream service calls, database queries all execute on a single thread assigned to the request.
In this demo project, we use thread pool Dispatchers.IO and async coroutine builder
to execute remote HTTP calls. The HTTP calls are non-blocking and suspendable. But this is
not ideal and should not be applied in real production. The reason is explained in following
sections.
Consider that we are providing an API to customers. And in order to serve the information we need to call a dozen of downstream services and also issue a dozen of database queries. If we make requests and queries in a sequential way, then the latency could become intolerable. As these requests and queries are not dependent, we decide to make parallel calls.
There’s nothing stops you from creating your own thread pool in frameworks like Spring. We could spawn a thread pool and then assign the downstream calls to it. So that calls are parallelized, and the problem looks solved.
But it is not that simple. The first problem is how many threads should the thread pool have? is it fixed thread pool or elastic with lower and higher limits? If we create too many, we are wasting resources and if we create too few, the calls will still have to wait for threads to complete the previously submitted tasks.
And the second problem is that should we create a thread pool when receiving an incoming request or when the app starts up? Neither way is ideal, because the first one would consume resources, and we need to take care of shutting down. For the second, we need to remember that when there are a high volume of calls from customers at the same time, the thread pool is not helpful because the tasks are just queued and waiting to be picked up.
Retrofit supports suspend functions
@GET
suspend fun getUsers(): List<User>But we are interested in NOT only the happy path result but also the failed requests' HTTP status codes and header information.
This should not be applied ⛔ in serious product.
We can also return Response type with suspend function ✅:
@GET("users")
suspend fun getUsers(): Response<List<User>>Retrofit original response type is Callable<T>
@GET("users")
fun getUsers(): Callable<List<User>>The synchronous method execute() on Callable will block executing threads and should be avoided.
We need to use the enqueue() method and craft customised code to utilise asynchronous calls. ✅
We can also use Retrofit RxJava extension to make remote calls reactive ✅
Spring WebFlux is an alternative to Spring Web. Spring Web is built to work with web containers e.g. Tomcat. It is a solution dates back to Java EE, J2EE, JSP and Servlets.
WebFlux is not a framework on top of Web or extra additional feature to Web. It’s another web framework that can replace Web. And it’s build with Reactive Streams in mind, with Netty as it’s underlying network server to take advantage of the Event Loops and the Project Reactor, and thus no Java EE web containers are used anymore.
You may already notice that our concurrency problem roots in blocking I/O, that means remote calls are just blocking the threads and waiting for remote response without doing anything.
WebFlux needs non-blocking Web clients and database drivers. Take non-blocking web client as an example, when it sends out a request, it yields the thread and when the response arrives it will resume execution and in this way the thread will never be blocked. Spring WebClient and R2DBC are non-blocking. MongoDB and PostgresDB provide non-blocking drivers.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent