Comments (5)
Hi,
DSL-JSON should work fine with high concurrent workloads. There are even few tweaks you can do to avoid thread locals and gain some additional performance.
But if you are hitting some limits, its most likely some Java/GC related issues.
And in that case its always best to profile whats going on and see where the bottleneck is. Its unlikely its in DSL-JSON, but if is there... we can certainly look how to fix/work around it.
from dsl-json.
Thanks for the response @zapov.
Can you please write down some of the tweaks you think will improve performance?
Feel free to link another post or example code.
I have done some profiling in which I saw a hotspot/bottleneck in the readContent method in DslJsonConverter class. With 1 TPS, deserialisation speeds are < 100ms, however with 30+ TPS, I see times of 1-3 seconds to deserialise.
Note, this is deserilisation of a 40MB byte array. 16GB RAM, 2VCPU. If you wish, I can share further benchmarking statistics.
from dsl-json.
You can find various suggestions in the readme but I doubt that will help you.
It sounds like your deserialization takes too long and you have too few CPUs to parallelize it.
Usual suggestions are to:
- use byte[] instead of some "smart" structure like ByteBuffer
- reuse this byte[]/streams instead of allocating them
- prefer local JsonReaders instead of thread locals
- avoid strings in favor of more exact data types
but nothing from that will help if you if your single deserialization takes too long and produces too much garbage.
Anyway... I doubt this has anything to do with DSL-JSON but you can paste YourKit profile or something like that.
from dsl-json.
I have added an example of a Profiler CPU Capture. As you can see the read/readContent are the hotspots.
As stated above, with 1 or 2 Threads concurrently running and executing deserialise every second, there is excellent performance < 100ms. When increasing this to 30 in running in ECS Fargate, it takes 1-3 seconds for each deserialisation operation.
from dsl-json.
If this is 2 CPU machine I would assume 2-4 threads would be optimal size, after that context switching might be hurting you too much.
So I would look into slowly scaling it and seeing when there is spike in performance degradation.
8% in getNextToken sounds like reading byte is taking too much time which I would assume is mostly switching this 40 MB structure across threads and reading byte off it.
Anyway, good luck with performance profiling... and figuring out where this performance bottleneck is coming from. If you need my further help you can ask for consulting contract via email.
from dsl-json.
Related Issues (20)
- About json field policy of object HOT 2
- Can't generate dsl for some kotlin data classes HOT 1
- Dependency org.postgresql:postgresql, leading to CVE problem HOT 1
- Question: Testing File Read with dsl-json HOT 2
- serialize/deserialize Set as a List HOT 2
- Question - Is possible to use array format with discriminator in abstract classes HOT 2
- Stack overflow error caused by dsl-json serialization Map HOT 2
- Polimorphic adapters HOT 9
- @CompiledJson annotation does not bind byte[] to BinaryConverter in v2.0.0 HOT 2
- Failure to serialize the basic generic objects. HOT 4
- support for kotlin value classes HOT 5
- JsonValue syntax tree HOT 7
- Filed filtering? HOT 4
- Re-using reader/writer. Make reset method public HOT 3
- Support reader.next(Class) inside a converter HOT 3
- Is there any support for deserialiazing from already parsed data? HOT 2
- Runtime reflection doesn't handle primitive boolean variables correctly HOT 4
- Configure for external classes using configuration HOT 1
- bug in `writerBinary` method of class `JsonWriter` HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dsl-json.