pipelining 100x slower on cache misses about olric HOT 10 CLOSED

Hey, @derekperkins Could you please share your configuration? I particularly need to learn the following fields:

• ReplicaCount
• ReadQuorum
• ReadRepair

from olric.

I'm not setting any of those explicitly, so they should all be 1 from the lan defaults

cfg := config.New("lan") // default configuration
cfg.ServiceDiscovery = map[string]any{
	"plugin": k8sDisc,
}
cfg.ReplicationMode = config.AsyncReplicationMode
cfg.LogLevel = "WARN"
cfg.LogVerbosity = 1

cfg.DMaps.Custom[name] = config.DMap{
	EvictionPolicy:  config.LRUEviction,
	MaxIdleDuration: 24 * time.Hour,
	MaxInuse:        50_000_000,
}

from olric.

FWIW, we're caching immutable objects, so stale data isn't possible. We're strictly focused on performance, and can set any knobs to whatever you suggest

from olric.

It may be related to #201. Initially, I thought we could add an option to disable the Last-Write-Wins policy but ReplicaCount is 1 by default. So it's already useless for your configuration.

from olric.

I wasn't doing any Put() calls in the pipeline, just Get(), but there would have been a lot of concurrent Put calls happening outside the Pipeline

from olric.

And after Exec() returns, Result() isn't making any network calls, correct? Those should have all been resolved inside Exec

from olric.

And after Exec() returns, Result() isn't making any network calls, correct? Those should have all been resolved inside Exec

That's correct.

I checked the code but I couldn't find any reason why cache misses are slow while hits are quite fast. Are cache misses slow with the normal DMap API? The related function is here:

from olric.

Are cache misses slow with the normal DMap API?

I don't believe so. We first implemented olric without using pipelines or goroutines for Get calls, which did cause a slowdown (deployment happened at metric spike) compared to the previous local only cache (ristretto), but nothing unexpected for ~100 sequential network calls.

It stabilized at those p-values until we deployed with pipelines (deployment at metric spike), which sent latency through the roof, even though it should have been fundamentally making the same ~100 requests with concurrency of 1.

from olric.

I just redeployed v0.5.1 and it shows massive improvements over the original trace above, but still is surprisingly slow on cache misses. They aren't taking 100ms now, but calling Result() on cache misses is taking ~1 ms, where cache hits take only ~0.02 ms

Long pipeline.Exec time, almost all cache hits

Short pipeline.Exec time, almost all cache misses

from olric.

Never mind, that was my bad. I wasn't ignoring the KeyNotFound error, so that handling was taking extra long, skewing the results. 😬

from olric.