Comments (10)
@sreuland Are you able to open a PR or share a diff of what would change in the quickstart image to use this different setup you were proposing? I suspect when I tested it I was not using the configuration you proposed.
from quickstart.
Ok, I got similar results to what I mentioned the first time I tested this (#490 (comment)):
Running without bucket list change:
Running with bucket list change:
The differences appear to be:
- CPU usage is more choppy and generally a bit higher when bucket list is enabled.
- Memory usage is a tiny bit lower when using the bucket list.
from quickstart.
@sreuland I gave this a go by enabling bucketlistdb on Horizon's captive core in the quickstart image. CPU usage was higher by ~5%. It's unclear to me if this is a better outcome as I couldn't see any other measure of improvement. There's a few resources we could measure impact on, but increased CPU seems pretty bad for devices that run on batteries like laptops.
What sort of performance improvements should we see on captive-core?
Does captive-core use on disk db? I thought it'd be in memory anyway?
Could you share some examples of the performance improvement you see?
from quickstart.
Hello @leighmcculloch ,
What sort of performance improvements should we see on captive-core?
when the captive core db is located on disk, which is done in quickstart by default for horizon, the savings are in less round-trips to the db from core process, as the bucklist acts like a read cache first, it was enabled on horizon cc config by default on horizon/4733 whenever 'use disk' is also enabled via CAPTIVE_CORE_USE_DB=true
which is the default horizon prefers to use for captive core config rather than 'in memory', horizon/4733 went in as of release horizon 2.24.0
Does captive-core use on disk db? I thought it'd be in memory anyway?
yes, for horizon cc in quickstart, it runs on disk mode,
Could you share some examples of the performance improvement you see?
@urvisavla , has actually done some recent horizon benchmarking for docs updates which included cc on disk and bucketlistdb which may provide more data. the findings are posted here - stellar/go#4960.
from quickstart.
i'm trying out two runs locally using --pubnet to observe, one with current quickstart:testing from dockerhub which has no bucketlist enabled, and the other built with this config, will watch activity monitor and see if any spikes on cpu/ram
horizon attempts to enable bucketlist by default, but it does so based on core's version string, looking for a semver of 19.6 or above, and since horizon built from source on quickstart image, I think it's not getting this automatically, so would have to add the enablement directly in the core .cfg
from quickstart.
I gave this a whirl using testnet, which I think is more representative of a developers use of quickstart. Pubnet is more data than a user is likely to use. The difference between the two doesn't seem that meaningful. Here's the resource usage of the docker container:
The only differences I see from this run is that:
- the bucket list resulted in more disk read/writes
- the bucket list resulted in less cpu usage
Seems fine to move ahead with this change then.
from quickstart.
ok, when you run quickstart are you including --enable-horizon-captive-core
, I realized that by default, and w/o including that flag, quickstart will launch horizon ingestion with --stellar-core-db-url
which skips captive core. maybe in tandem to adding bucketlist to captive configs in quickstart, we take the same time to just remove --enable-horizon-captive-core
so that horizon always uses captive core, and avoid confusion, i hacked it out in the sample pr - 681cc5d
docker run --rm -it -p "8000:8000" --name stellar stellar/quickstart:testing --pubnet --enable-horizon-captive-core
from quickstart.
Ah, I missed that. Thanks. When we merge #498 then this won't be such an easy mistake to make. I'll rerun and see what it is like on my system with it enabled.
from quickstart.
cool, #498 looks good, maybe can just add the bucketlist enablement in there as it's closely related to captive core anyways? - ac9088a
I ran with --testnet --enable-horizon-captive-core
and bucketlistdb enabled and disabled, nothing spiked in resources on either, running with --pubnet and bucketlistdb, it did elevate on CPU to max and stayed there when it got to applying:
stellar-core: Catching up; Catching up to ledger 48382143: Applying buckets 0%. Currently on level 10
, I need to try it with no bucketlistdb to see if any different.
from quickstart.
This issue is stale because it has been open for 30 days with no activity. It will be closed in 30 days unless the stale label is removed.
from quickstart.
Related Issues (20)
- Fails to start as a GitHub Action Service HOT 2
- Update GitHub Action builds to remove use of set-output and save-state HOT 5
- include soroban diagnostics on core tx sub response by default HOT 2
- Soroban settings upgrade failure is silent HOT 3
- What date quickstart docker image for stellar-core v20.1.0 will be released? HOT 2
- Automatically setup native token on quickstart local networks HOT 3
- Is public archive or quickstart image for v20 good now for horizon ingestion?! HOT 9
- Add support for ENABLE_DIAGNOSTICS_FOR_TX_SUBMISSION HOT 4
- RFC: Enable soroban-rpc by default HOT 2
- Stellar Quickstart not working HOT 20
- `soroban-rpc` doesn't start in `--pubnet` mode
- Consider not running validator node in network modes HOT 6
- `--limits unlimited` is limiting at least the `contractDataKeySizeBytes` limit HOT 3
- Add more "things are going well" feedback to the logs when only `--enable rpc` is running HOT 2
- Docker desktop SIGABRT error in containers log HOT 1
- Friendbot always returning 502 Bad Gateway using Stellar Quickstart HOT 5
- Pipeline not triggerred/retagged for dockerhub build on quickstart:latest?! HOT 3
- 'Error setting configUpgradeSet' with --local and CORE_REF=v21.1.0 HOT 2
- Add an explorer UI to the quickstart image HOT 4
- Support forking from mainnet (or any target network) HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from quickstart.