Comments (5)
Thanks for reporting this @deathy. Do you by any chance have a sample application that replicates the performance issues you're seeing?
from spring-boot.
Yes I'll try to create a proper sample in the next days. Might take some time, have to see with different numbers/sizes of files and need to see if it's the same with an in-memory database.
Meanwhile I have some numbers from debugging my app startup:
- 1823 files and 236 directories inside executable jar
- in
org.springframework.boot.loader.zip.VirtualZipDataBlock#VirtualZipDataBlock
constructor I have:- centralRecords.length = 1754
- parts.size = 14033
- ByteArrayDataBlock: 5263 items
- VirtualZipDataBlock$DataPart: 8770 items
- centralParts.size = 5262
- sizeOfCentralDirectory = 176218
Grouping the 14033 parts by DataBlock::size, top 5:
- size 0: 3508 blocks
- size 46: 1804 blocks
- size 30: 1776 blocks
- size 16: 1756 blocks
- size 2: 215 blocks
Also ran async-profiler with event=alloc:
from spring-boot.
Observed during debugging:
JDK JarInputStream/ZipInputStream called from hibernate loader read sequentially.
Reading sequentially next 512 or how ever many bytes (saw some 3 byte reads for iterating entries) re-incur the whole traversal that was already done since for (DataBlock part : this.parts)
inside org.springframework.boot.loader.zip.VirtualDataBlock#read
is a linear lookup?
It can't skip ahead even though pos
is passed in as param.
That matches with java/util/ImmutableCollections$ListItr.next
appearing in profile.
from spring-boot.
@deathy just to double check, are you using the latest 3.2.x release?
from spring-boot.
@philwebb yes I'm on 3.2.4 with original application.
I finally managed to make a sample: https://github.com/deathy/spring-startup-test
Has a bunch of randomly generated files and basic config to trigger visible difference (although much lower than original app since it's simpler). Measured startup times are in README there.
And through it discovered another condition of this happening: actually having the hibernate scanner active.
In automatic configuration since #15565 and #15321 the Hibernate Scanner is disabled by default.
But it is not disabled if you're actually using EntityManagerFactoryBuilder
manually unless you pass in specific property.
In my original app I added .properties(Map.of("hibernate.archive.scanner","org.hibernate.boot.archive.scan.internal.DisabledScanner"))
and that completely eliminated time from Hibernate Scanner.
There I also had two persistence contexts (separate DBs) so the scan was being performed twice.
I still see a significant difference compared to running unpacked but 2-3X not 4-5X and profiling so far hasn't shown any big obvious block like before, everything is more spread out.
The sequential iteration through parts
in VirtualDataBlock#read
still feels wrong but I have a workaround for scanner and not sure anything else uses it that heavily.
Hibernate Scanner questions:
- should
JpaBaseConfiguration#entityManagerFactory
andEntityManagerFactoryBuilder
have the same default meaning disabling Hibernate Scanner?- if not, maybe potential footgun should be documented somewhere?
from spring-boot.
Related Issues (20)
- Upgrade to Spring Authorization Server 1.1.7
- Upgrade to Spring Data Bom 2023.0.11
- Upgrade to Spring Framework 6.0.19
- Upgrade to Spring Integration 6.1.8
- Upgrade to Spring LDAP 3.1.5
- Upgrade to Spring Security 6.1.9
- Upgrade to Neo4j Java Driver 5.19.0
- SpringBootExceptionHandler thread local leak reported by Tomcat
- Upgrade to Micrometer 1.12.5
- Upgrade to Micrometer Tracing 1.2.5
- Upgrade to Reactor Bom 2023.0.5
- Upgrade to Spring AMQP 3.1.4
- Upgrade to Spring Authorization Server 1.2.4
- Upgrade to Spring Data Bom 2023.1.5
- Upgrade to Spring Framework 6.1.6
- Upgrade to Spring Integration 6.2.4
- Upgrade to Spring LDAP 3.2.3
- Upgrade to Spring Pulsar 1.0.5
- Upgrade to Spring Security 6.2.4
- Upgrade to Neo4j Java Driver 5.19.0
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from spring-boot.