Comments (10)
Yeah, we shouldn't fiddle with the Undertow executors.
The default one produces this plot:
As soon as we meddle with it, it starts breaking.
With virtual threads:
With virtual threads and concurrency throttle to 200:
With platform threads and concurrency throttle to 200:
It breaks in all configurations when setting the executor.
I'm going to revert the virtual thread support for Undertow.
from spring-boot.
That's one of the drawbacks when moving from a thread pool to virtual threads per task without a pool. When using thread pools, they usually have an upper bound for the size, limiting the maximum resource consumption. Unpooled virtual threads don't have that. I assume the application you're benchmarking is a simple demo application which just returns some hardcoded data? Because usually, if there's no pool in the webserver, some other pool in the application is limiting the throughput (e.g. the connection pool to the database).
Btw, you don't see the memory usage in VisualVM, as this is direct allocated memory, which is off-heap.
from spring-boot.
Yes, it's the Spring Boot demo project with a basic GET endpoint with no return value. Sorry, could you explain the cause for this again? Aren't virtual threads still limited by the size of the platform threads in the carrier pool? The Tomcat and Jetty servlets don't seem to have the same issue when using virtual threads. Also, thank you for the explanation about the profiler.
from spring-boot.
One possibility could be this:
1. VT1: Start accepting the request
2. VT1: Allocate memory
3. VT1: Block on something (maybe reading http from the TCP socket)
4. VT1: Handle request
5. VT1: Free memory
With enough requests lined up, the block at 3. could be the reason why the memory is exhausted. At this block, the scheduler frees the underlying platform thread, and switches to a new virtual thread, which allocates memory, blocks, a new virtual thread is scheduled, allocates memory, etc etc.
from spring-boot.
You could use a SimpleAsyncTaskExecutor
with setConcurrencyLimit
to limit the maximum allowed number of virtual threads.
from spring-boot.
as long as I remember, there is an issue that talks about virtual threads in Spring Boot here:
#38819
there is one part I saw that says in version 3.3.x the usage of the virtual thread must automatically happen for undertow. so can it be related to this issue?
can u check if this also happens with version 3.2.x ? @Tythor
from spring-boot.
@MetaiR
Yes, I can confirm the same behavior happens in v3.2.3. Since spring.threads.virtual.enabled=true
does not apply to Undertow servlets until v3.3.0, I had to use the undertowDeploymentInfoCustomizer
to enable virtual threading.
@mhalbritter
I attempted to use a SimpleAsyncTaskExecutor
with virtual threads with a concurrency limit of 10, but the issue still persisted.
@Bean
public UndertowDeploymentInfoCustomizer undertowDeploymentInfoCustomizer() {
return deploymentInfo -> {
SimpleAsyncTaskExecutor simpleAsyncTaskExecutor = new SimpleAsyncTaskExecutor();
simpleAsyncTaskExecutor.setVirtualThreads(true);
simpleAsyncTaskExecutor.setConcurrencyLimit(10);
deploymentInfo.setExecutor(simpleAsyncTaskExecutor);
};
}
However, this led me to try using a non virtual threaded executor instead. Surprisingly, I observed the same behavior, albeit at a much slower growth rate of about 2m for 20GB instead of 30s. These three executors produced similar results:
deploymentInfo.setExecutor(new SimpleAsyncTaskExecutor());
deploymentInfo.setExecutor(Executors.newThreadPerTaskExecutor(Thread.ofPlatform().factory()));
deploymentInfo.setExecutor(Executors.newThreadPerTaskExecutor(Executors.defaultThreadFactory()));
However, when using
deploymentInfo.setExecutor(Executors.newCachedThreadPool());
the memory growth disappeared. Looking into the implementation, the main difference is that the cachedThreadPool executor has a keepAliveTime
of 60s, while the others have a keepAliveTime
of 0s.
Reducing the keepAliveTime
to as low as 1s did not cause the memory growth in my tests. But setting it to 0s reproduced the same issue.
deploymentInfo.setExecutor(new ThreadPoolExecutor(0, Integer.MAX_VALUE, 1L, TimeUnit.SECONDS, new SynchronousQueue<>()));
So it looks like this issue is not unique to virtual threads, but perhaps with the way Undertow is handling executors?
from spring-boot.
Yeah, looks like something is wrong here, but it doesn't seem to be in Spring Boot. Please open an issue on the Undertow tracker, and feel free to drop the link in this issue. Thanks!
from spring-boot.
We decided to reopen the issue. The out of box experience with virtual threads in undertow is bad, and if we (or the undertow team) can't fix the problems with it, we might remove virtual threads support for undertow again.
from spring-boot.
Okay, I was about to suggest disabling the auto configuration for spring.threads.virtual.enabled=true
in v3.3.x. This issue would likely be a problem that most users will not be aware of. Thanks!
cff1b33
from spring-boot.
Related Issues (20)
- Error in saving an entity that has a HashSet field when the @EnableReactiveMongoAuditing annotation is active. HOT 1
- org.thymeleaf.exceptions.TemplateInputException HOT 1
- spring data rest and jpa repository HOT 1
- Add CI for Java 22
- Add TWENTY_TWO to JavaVersion enum
- Document support for Java 22 HOT 1
- Add TWENTY_TWO to JavaVersion enum
- RedisHealthIndicator.doHealthCheck is not invoked when health check is performed by http://localhost:8080/actuator/health HOT 3
- gradlew bootBuildImage fails with Podman on macOS Sonoma HOT 13
- Exclude infrastructure beans from lazy initialization when lazy-initialization=true
- Official support for Flyway 10 HOT 6
- @JsonProperty annotation seems to be ignored when using query object parameter HOT 1
- Finish CDS work
- @ConfigurationProperties(prefix="application.id) is not working HOT 1
- Circular placeholder reference when running mvn clean -Pnative native:compile HOT 1
- Memory increases with 3.2.3 HOT 13
- Add public getUndertow method to UndertowWebServer HOT 6
- Configuration properties cannot be bound to DruidDataSource when the environment contains a non-enumerable property source HOT 4
- Incorrect Handling of Active Profiles in Docker Compose Integration HOT 5
- Incorrect classloader used by commom ForkJoinPool when using Executable Jar HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from spring-boot.