dragonflydb / dragonfly Goto Github PK

1. save should dump lua scripts
2. save should cover multiple database, currently saves only db0.

when changing this_fiber::yield(); with this_fiber::sleep_for(3us); in rdb_snapshot.cc
dragonfly sometimes deadlocks. Currently, it does not have any impact since we use yield() but it's worrisome that
the helio scheduler has deadlock bugs. I need to think how to reproduce it in synthetic test and fix it.

use redis_exporter for inspiration and for metric names. No need to export everything, at least those metrics that appear in grafana dashboard.

# Server
# Clients
# Memory
# Stats
# TIERED_STORAGE
# Replication
# Commandstats
# Errorstats
# Keyspace

But CPU is not listed.

del ztmp
zadd ztmp xx 10 x
should return 0
currently returns nil

REDIS
127.0.0.1:6380> del tmp
(integer) 1
127.0.0.1:6380> zadd tmp xx 10 x
(integer) 0

DRAGONFLY
127.0.0.1:6379> del tmp
(integer) 0
127.0.0.1:6379> zadd tmp xx 10 x
(nil)

To reproduce the missing functionality:

taskset -c 1 ./list_family_test --gtest_repeat=100 --gtest_filter=*ize --vmodule=transaction=2,engine_shard_set=1,list_family=1 --logbuflevel=-1

we should support:
a. version
b. git_sha
c. build timestamp

it should have optional dummy values for dev build and correct ones for release builds.

REDIS:
127.0.0.1:6380> del ztmp
(integer) 0
127.0.0.1:6380> zadd ztmp lt gt 10 x
(error) ERR GT, LT, and/or NX options at the same time are not compatible

DRAGONFLY:
127.0.0.1:6379> del ztmp
(integer) 0
127.0.0.1:6379> zadd ztmp lt gt 10 x
(integer) 1

REDIS
127.0.0.1:6380> SET foo bar EX -9999999999999999
(error) ERR invalid expire time in set

DRAGONFLY
127.0.0.1:6379> SET foo bar EX -9999999999999999
(error) ERR expiry is out of range

Hey Folks,

I just stumbled across this insane project - great work so far! 🚀

I was wondering if you guys are already working on a helm-Chart in order to deploy dragonfly to Kubernetes?
It would be a lot easier for us to test dragonfly in our environment then.

Best regards!

See https://ably.com/blog/redis-keys-do-not-expire-atomically for motivation.

if we use transactions to set expiry or to access multiple possibly expired items, we should use a consistent clock value during the transaction execution.

current behavior: we use real clock to set expiry and to test it.

./runtest --host localhost --port 6379 --tags -cluster:skip --ignore-encoding --ignore-digest --quiet --singledb --single unit/type/list --only "LINDEX against non-list value"

lindex mylist 0
should return WRONGTYPE but returns nil

Hello, came across Dragonfly on Hacker News. Very cool project – good luck with it!

I am very interested in Stream data structure support - think that's the only missing command for me to start experimenting with Dragonfly.

Specifically, in my use case I am using:

• XADD
• XRANGE
• XREVRANGE

Hope this will be added at some point.

./runtest --host localhost --port 6379 --tags -cluster:skip --ignore-encoding --ignore-digest --quiet --singledb --single unit/type/list --skiptest "check compression" --skiptest "LPOS|LMOV|BLMPOP|BRPOPLPUSH|RPOPLPUSH|LMPOP|CLIENT|STORE" --tags -needs:debug --skiptest "client unblock tests" --skiptest "List ziplist of various encodings"

[err]: LPOP/RPOP with the count 0 returns an empty array in RESP2 in tests/unit/type/list.tcl
Expected '0' to be equal to '-1' (context: type eval line 3 cmd {assert_equal {0} [r lpop listcount 0]} proc ::test)
[err]: LPOP/RPOP with against non existing key in RESP2 in tests/unit/type/list.tcl
Expected '$-1' to be equal to '-1' (context: type proc line 3 cmd {assert_equal $response $resp2_response} proc ::verify_resp_response level 2)

on part with Redis 2.8 -> on par with Redis 2.8 in the README

and there are two instances of "veticly" on https://dragonflydb.io/

Nice project! :-)

from redis_exporter log:

ERRO[0004] Couldn't set client name, err: ERR unknown command `CLIENT` 
ERRO[0004] WARNING, LOGGED ONCE ONLY: cmd LATENCY LATEST, err: ERR unknown command `LATENCY` 
ERRO[0009] Couldn't set client name, err: ERR unknown command `CLIENT` 

debian bullseye is quite popular. Apparently, its kernel version is 5.10. We should support it.
apparently there docker run has limited amount of mlocked. We should gracefully explain how to fix it as well.

./runtest --host localhost --port 6379 --tags -cluster:skip --ignore-encoding --ignore-digest --quiet --singledb --single unit/multi --only "MULTI / EXEC basics"

I20220407 17:08:08.921078 226477 main_service.cc:361] Got: [DEBUG,log,### Starting test MULTI / EXEC basics in tests/unit/multi.tcl]
I20220407 17:08:08.921932 226476 main_service.cc:361] Got: [DEL,mylist]
I20220407 17:08:08.922478 226476 main_service.cc:361] Got: [RPUSH,mylist,a]
I20220407 17:08:08.922977 226476 main_service.cc:361] Got: [RPUSH,mylist,b]
I20220407 17:08:08.923432 226476 main_service.cc:361] Got: [RPUSH,mylist,c]
I20220407 17:08:08.923820 226476 main_service.cc:361] Got: [MULTI]
I20220407 17:08:08.924144 226476 main_service.cc:361] Got: [LRANGE,mylist,0,-1]
I20220407 17:08:08.924310 226476 main_service.cc:361] Got: [PING]
I20220407 17:08:08.924551 226476 main_service.cc:361] Got: [EXEC]
F20220407 17:08:08.925173 226476 transaction.cc:151] Check failed: args.size() > 1U (1 vs. 1)
*** Check failure stack trace: ***
@ 0x5654c215cddf google::LogMessage::Fail()
@ 0x5654c215cd21 google::LogMessage::SendToLog()
@ 0x5654c215c605 google::LogMessage::Flush()
@ 0x5654c2160164 google::LogMessageFatal::~LogMessageFatal()
@ 0x5654c17b7b38 dfly::Transaction::InitByArgs()
@ 0x5654c1572d34 dfly::Service::Exec()
@ 0x5654c1578d0b ZZN4dfly7Service16RegisterCommandsEvENKUlN4absl12lts_202111024SpanINS3_IcEEEEPNS_17ConnectionContextEE2_clES5_S7
@ 0x5654c15b57c0 ZN3fu27abi_4006detail10invocation6invokeIRKZN4dfly7Service16RegisterCommandsEvEUlN4absl12lts_202111024SpanINS8_IcEEEEPNS4_17ConnectionContextEE2_JSA_SC_EEEDTclcl7forwardIT_Efp_Espcl7forwardIT0_Efp0_EEEOSG_DpOSH
@ 0x5654c15ae321 ZN3fu27abi_4006detail12type_erasure16invocation_table14function_traitIKFvN4absl12lts_202111024SpanINS7_IcEEEEPN4dfly17ConnectionContextEEE16internal_invokerINS2_3boxILb1EZNSA_7Service16RegisterCommandsEvEUlS9_SC_E2_SaISI_EEELb1EE6invokeEPKNS2_13data_accessorEmS9_SC
@ 0x5654c15e0110 ZNK3fu27abi_4006detail12type_erasure6tables6vtableINS1_8propertyILb0ELb0EJKFvN4absl12lts_202111024SpanINS8_IcEEEEPN4dfly17ConnectionContextEEEEEE6invokeILm0EJPKNS2_13data_accessorERKmSA_SD_EEEDcDpOT0
@ 0x5654c15e045c ZN3fu27abi_4006detail12type_erasure7erasureILb1ENS1_6configILb1ELb1ENS_16capacity_defaultEEENS1_8propertyILb0ELb0EJKFvN4absl12lts_202111024SpanINSA_IcEEEEPN4dfly17ConnectionContextEEEEEE6invokeILm0ERKSI_JSC_SF_EEEDcOT0_DpOT1
@ 0x5654c15e0679 fu2::abi_400::detail::type_erasure::invocation_table::operator_impl<>::operator()()
@ 0x5654c15cd056 dfly::CommandId::Invoke()
@ 0x5654c1563c4d dfly::Service::DispatchCommand()
@ 0x5654c1ba5874 facade::Connection::ParseRedis()
@ 0x5654c1ba8f5c facade::Connection::IoLoop()
@ 0x5654c1ba2c49 facade::Connection::ConnectionFlow()
@ 0x5654c1b9f96c facade::Connection::HandleRequests()
@ 0x5654c1f8846c util::ListenerInterface::RunSingleConnection()
@ 0x5654c1f82bbb _ZZN4util17ListenerInterface13RunAcceptLoopEvENKUlvE0_clEv
@ 0x5654c1fa0e02 ZSt13__invoke_implIvZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_JEET_St14__invoke_otherOT0_DpOT1
@ 0x5654c1f9dec7 ZSt8__invokeIZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_JEENSt15__invoke_resultIT_JDpT0_EE4typeEOS4_DpOS5
@ 0x5654c1f98c3b _ZSt12__apply_implIZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_St5tupleIJEEJEEDcOT_OT0_St16integer_sequenceImJXspT1_EEE
@ 0x5654c1f98cb2 ZSt5applyIZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_St5tupleIJEEEDcOT_OT0
@ 0x5654c1f98fd1 _ZN5boost6fibers14worker_contextIZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_JEE4run_EONS_7context5fiberE
@ 0x5654c1fb60be ZSt13__invoke_implIN5boost7context5fiberERMNS0_6fibers14worker_contextIZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_JEEEFS2_OS2_ERPS8_JS2_EET_St21__invoke_memfun_derefOT0_OT1_DpOT2
@ 0x5654c1fb4f80 ZSt8__invokeIRMN5boost6fibers14worker_contextIZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_JEEEFNS0_7context5fiberEOS8_EJRPS6_S8_EENSt15__invoke_resultIT_JDpT0_EE4typeEOSG_DpOSH
@ 0x5654c1fb4469 _ZNSt5_BindIFMN5boost6fibers14worker_contextIZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_JEEEFNS0_7context5fiberEOS8_EPS6_St12_PlaceholderILi1EEEE6__callIS8_JS9_EJLm0ELm1EEEET_OSt5tupleIJDpT0_EESt12_Index_tupleIJXspT1_EEE
@ 0x5654c1fb2be5 ZNSt5_BindIFMN5boost6fibers14worker_contextIZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_JEEEFNS0_7context5fiberEOS8_EPS6_St12_PlaceholderILi1EEEEclIJS8_ES8_EET0_DpOT
@ 0x5654c1fafd93 ZSt13__invoke_implIN5boost7context5fiberERSt5_BindIFMNS0_6fibers14worker_contextIZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_JEEEFS2_OS2_EPS9_St12_PlaceholderILi1EEEEJS2_EET_St14__invoke_otherOT0_DpOT1
@ 0x5654c1fad979 ZSt8__invokeIRSt5_BindIFMN5boost6fibers14worker_contextIZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_JEEEFNS1_7context5fiberEOS9_EPS7_St12_PlaceholderILi1EEEEJS9_EENSt15__invoke_resultIT_JDpT0_EE4typeEOSK_DpOSL
@ 0x5654c1fa962c ZSt6invokeIRSt5_BindIFMN5boost6fibers14worker_contextIZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_JEEEFNS1_7context5fiberEOS9_EPS7_St12_PlaceholderILi1EEEEJS9_EENSt13invoke_resultIT_JDpT0_EE4typeEOSK_DpOSL
*** SIGABRT received at time=1649340488 on cpu 0 ***
PC: @ 0x7f162c45e03b (unknown) raise
@ 0x5654c21a56f2 64 absl::lts_20211102::WriteFailureInfo()
@ 0x5654c21a58f7 96 absl::lts_20211102::AbslFailureSignalHandler()
@ 0x7f162c6213c0 (unknown) (unknown)
@ 0x5654c215cddf 16 google::LogMessage::Fail()
@ 0x5654c215cd21 144 google::LogMessage::SendToLog()
@ 0x5654c215c605 80 google::LogMessage::Flush()
@ 0x5654c2160164 32 google::LogMessageFatal::~LogMessageFatal()
@ 0x5654c17b7b38 1504 dfly::Transaction::InitByArgs()
@ 0x5654c1572d34 688 dfly::Service::Exec()
@ 0x5654c1578d0b 192 dfly::Service::RegisterCommands()::{lambda()#4}::operator()()
@ 0x5654c15b57c0 80 fu2::abi_400::detail::invocation::invoke<>()
@ 0x5654c15ae321 288 fu2::abi_400::detail::type_erasure::invocation_table::function_trait<>::internal_invoker<>::invoke()
@ 0x5654c15e0110 128 fu2::abi_400::detail::type_erasure::tables::vtable<>::invoke<>()
@ 0x5654c15e045c 240 fu2::abi_400::detail::type_erasure::erasure<>::invoke<>()
@ 0x5654c15e0679 240 fu2::abi_400::detail::type_erasure::invocation_table::operator_impl<>::operator()()
@ 0x5654c15cd056 192 dfly::CommandId::Invoke()
@ 0x5654c1563c4d 2336 dfly::Service::DispatchCommand()
@ 0x5654c1ba5874 608 facade::Connection::ParseRedis()
@ 0x5654c1ba8f5c 528 facade::Connection::IoLoop()
@ 0x5654c1ba2c49 656 facade::Connection::ConnectionFlow()
@ 0x5654c1b9f96c 1088 facade::Connection::HandleRequests()
@ 0x5654c1f8846c 416 util::ListenerInterface::RunSingleConnection()
@ 0x5654c1f82bbb 48 util::ListenerInterface::RunAcceptLoop()::{lambda()#2}::operator()()
@ 0x5654c1fa0e02 48 std::__invoke_impl<>()
@ 0x5654c1f9dec7 32 std::__invoke<>()
@ 0x5654c1f98c3b 32 std::_apply_impl<>()
@ 0x5654c1f98cb2 48 std::apply<>()
@ 0x5654c1f98fd1 240 boost::fibers::worker_context<>::run()
@ 0x5654c1fb60be 96 std::__invoke_impl<>()
@ 0x5654c1fb4f80 96 std::__invoke<>()
@ 0x5654c1fb4469 224 std::_Bind<>::__call<>()
@ 0x5654c1fb2be5 176 std::_Bind<>::operator()<>()
@ ... and at least 6 more frames
Aborted (core dumped)

We use base + offset computation for expiry, where offset is 32bit value that could hold milliseconds period.
In practice we are limited to ~36 hours before the offset overflows. We need to implement a per-hour process that updates the offsets in the expire table.

./runtest --host localhost --port 6379 --tags -cluster:skip --ignore-encoding --quiet --singledb --single unit/type/incr --skiptest "INCR uses shared objects" --skiptest "INCR can modify objects"
[err]: INCRBYFLOAT fails against key with spaces (left) in tests/unit/type/incr.tcl
Expected 'ERRvalid' to equal or match '12'
[err]: INCRBYFLOAT fails against key with spaces (right) in tests/unit/type/incr.tcl
Expected 'ERRvalid' to equal or match '12'
[err]: INCRBYFLOAT fails against key with spaces (both) in tests/unit/type/incr.tcl
Expected 'ERRvalid' to equal or match '12'
[err]: INCRBYFLOAT does not allow NaN or Infinity in tests/unit/type/incr.tcl
Expected 'ERRwould produce' to equal or match 'ERR value is not a valid float'
[1/1 done]: unit/type/incr (0 seconds)

               The End

Execution time of different units:
0 seconds - unit/type/incr

!!! WARNING The following tests failed:

*** [err]: INCRBYFLOAT fails against key with spaces (left) in tests/unit/type/incr.tcl
Expected 'ERRvalid' to equal or match '12'
*** [err]: INCRBYFLOAT fails against key with spaces (right) in tests/unit/type/incr.tcl
Expected 'ERRvalid' to equal or match '12'
*** [err]: INCRBYFLOAT fails against key with spaces (both) in tests/unit/type/incr.tcl
Expected 'ERRvalid' to equal or match '12'
*** [err]: INCRBYFLOAT does not allow NaN or Infinity in tests/unit/type/incr.tcl
Expected 'ERRwould produce' to equal or match 'ERR value is not a valid float'

REDIS:
127.0.0.1:6380> info cpu
# CPU
used_cpu_sys:70.619133
used_cpu_user:38.926553
used_cpu_sys_children:0.000000
used_cpu_user_children:0.007828
used_cpu_sys_main_thread:70.617346
used_cpu_user_main_thread:38.928299

DRAGONFLY:
127.0.0.1:6379> info cpu

1. SCAN - TYPE / COUNT / MATCH
2. SET - KEEPTTL / GET / EXAT / PXAT
3. EXPIRE - NX|XX|GT|LT

ERR unsupported option LIMIT

currently we only support grow path and a shrinkage with Clear()

i'm from hedgehog.vc (and i'm a cofounder of posthog.com)... we invest $100k to $300k+ in open source startups as a fun way to embrace other open source projects, and to help you learn from us

i can see there is a company formed around the project, and i read on HN that you're bootstrapped. i was curious if you'd consider taking funding or if you'd just like to talk about it?

if interested, id love to chat - james at posthog dot com, if not feel free to close this issue - hope it's ok to ask like this

REDIS
127.0.0.1:6380> set foo bar
OK
127.0.0.1:6380> EXPIRE foo 9223370399119966
(error) ERR invalid expire time in expire

DRAGONFLY:
127.0.0.1:6379> set foo bar
OK
127.0.0.1:6379> EXPIRE foo 9223370399119966
(error) ERR expiry is out of range

another case:
REDIS
127.0.0.1:6380> EXPIRE foo -9223372036854776
(error) ERR invalid expire time in expire

DRAGONFLY:
127.0.0.1:6379> EXPIRE foo -9223372036854776
(integer) 1

REDIS:
127.0.0.1:6380> zadd tmp 0.123123123 x
(integer) 1
127.0.0.1:6380> zscore tmp x
"0.123123123"

DRAGONFLY:
127.0.0.1:6379> zadd tmp 0.123123123 x
(integer) 1
127.0.0.1:6379> zscore tmp x
"0.123123"

This looks really interesting! Have you considered benchmarking against Skytable (which has very impressive benchmarks surpassing both KeyDB and Redis by a large factor) or Aerospike (which also has very impressive benchmarks against Redis)? I have been following all the projects lately and am just curious.

PS: I know Aerospike is more mature but Skytable is a pretty recent project

I20220407 14:32:14.753866 217703 main_service.cc:361] Got: [DEBUG,log,### Starting test Vararg DEL in tests/unit/keyspace.tcl]
I20220407 14:32:14.754456 217705 main_service.cc:361] Got: [SET,foo1{t},a]
I20220407 14:32:14.755033 217705 main_service.cc:361] Got: [SET,foo2{t},b]
I20220407 14:32:14.755450 217705 main_service.cc:361] Got: [SET,foo3{t},c]
I20220407 14:32:14.755872 217705 main_service.cc:361] Got: [DEL,foo1{t},foo2{t},foo3{t},foo4{t}]
I20220407 14:32:14.756801 217705 main_service.cc:361] Got: [MGET,foo1{t},foo2{t},foo3{t}]
==217701==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7fcfd660e180; bottom 0x7fcfc6e73000; size: 0x00000f79b180 (259633536)
False positive error reports may follow
For details see google/sanitizers#189
F20220407 14:32:14.757282 217705 transaction.cc:978] Check failed: 'EngineShard::tlocal()' Must be non NULL
*** Check failure stack trace: ***
@ 0x5565e2540ddf google::LogMessage::Fail()
@ 0x5565e2540d21 google::LogMessage::SendToLog()
@ 0x5565e2540605 google::LogMessage::Flush()
@ 0x5565e2544164 google::LogMessageFatal::~LogMessageFatal()
@ 0x5565e1bece27 google::CheckNotNull<>()
@ 0x5565e1bbe131 dfly::Transaction::ShardArgsInShard()
@ 0x5565e1b62c8b dfly::StringFamily::MGet()
@ 0x5565e1a2fc0e ZN3fu27abi_4006detail10invocation6invokeIRKPFvN4absl12lts_202111024SpanINS6_IcEEEEPN4dfly17ConnectionContextEEJS8_SB_EEEDTclcl7forwardIT_Efp_Espcl7forwardIT0_Efp0_EEEOSG_DpOSH
@ 0x5565e1a2aeba fu2::abi_400::detail::type_erasure::invocation_table::function_trait<>::internal_invoker<>::invoke()
@ 0x5565e19c4110 ZNK3fu27abi_4006detail12type_erasure6tables6vtableINS1_8propertyILb0ELb0EJKFvN4absl12lts_202111024SpanINS8_IcEEEEPN4dfly17ConnectionContextEEEEEE6invokeILm0EJPKNS2_13data_accessorERKmSA_SD_EEEDcDpOT0
@ 0x5565e19c445c ZN3fu27abi_4006detail12type_erasure7erasureILb1ENS1_6configILb1ELb1ENS_16capacity_defaultEEENS1_8propertyILb0ELb0EJKFvN4absl12lts_202111024SpanINSA_IcEEEEPN4dfly17ConnectionContextEEEEEE6invokeILm0ERKSI_JSC_SF_EEEDcOT0_DpOT1
@ 0x5565e19c4679 fu2::abi_400::detail::type_erasure::invocation_table::operator_impl<>::operator()()
@ 0x5565e19b1056 dfly::CommandId::Invoke()
@ 0x5565e1947c4d dfly::Service::DispatchCommand()
@ 0x5565e1f89874 facade::Connection::ParseRedis()
@ 0x5565e1f8cf5c facade::Connection::IoLoop()
@ 0x5565e1f86c49 facade::Connection::ConnectionFlow()
@ 0x5565e1f8396c facade::Connection::HandleRequests()
@ 0x5565e236c46c util::ListenerInterface::RunSingleConnection()
@ 0x5565e2366bbb _ZZN4util17ListenerInterface13RunAcceptLoopEvENKUlvE0_clEv
@ 0x5565e2384e02 ZSt13__invoke_implIvZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_JEET_St14__invoke_otherOT0_DpOT1
@ 0x5565e2381ec7 ZSt8__invokeIZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_JEENSt15__invoke_resultIT_JDpT0_EE4typeEOS4_DpOS5
@ 0x5565e237cc3b _ZSt12__apply_implIZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_St5tupleIJEEJEEDcOT_OT0_St16integer_sequenceImJXspT1_EEE
@ 0x5565e237ccb2 ZSt5applyIZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_St5tupleIJEEEDcOT_OT0
@ 0x5565e237cfd1 _ZN5boost6fibers14worker_contextIZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_JEE4run_EONS_7context5fiberE
@ 0x5565e239a0be ZSt13__invoke_implIN5boost7context5fiberERMNS0_6fibers14worker_contextIZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_JEEEFS2_OS2_ERPS8_JS2_EET_St21__invoke_memfun_derefOT0_OT1_DpOT2
@ 0x5565e2398f80 ZSt8__invokeIRMN5boost6fibers14worker_contextIZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_JEEEFNS0_7context5fiberEOS8_EJRPS6_S8_EENSt15__invoke_resultIT_JDpT0_EE4typeEOSG_DpOSH
@ 0x5565e2398469 _ZNSt5_BindIFMN5boost6fibers14worker_contextIZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_JEEEFNS0_7context5fiberEOS8_EPS6_St12_PlaceholderILi1EEEE6__callIS8_JS9_EJLm0ELm1EEEET_OSt5tupleIJDpT0_EESt12_Index_tupleIJXspT1_EEE
@ 0x5565e2396be5 ZNSt5_BindIFMN5boost6fibers14worker_contextIZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_JEEEFNS0_7context5fiberEOS8_EPS6_St12_PlaceholderILi1EEEEclIJS8_ES8_EET0_DpOT
@ 0x5565e2393d93 ZSt13__invoke_implIN5boost7context5fiberERSt5_BindIFMNS0_6fibers14worker_contextIZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_JEEEFS2_OS2_EPS9_St12_PlaceholderILi1EEEEJS2_EET_St14__invoke_otherOT0_DpOT1
@ 0x5565e2391979 ZSt8__invokeIRSt5_BindIFMN5boost6fibers14worker_contextIZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_JEEEFNS1_7context5fiberEOS9_EPS7_St12_PlaceholderILi1EEEEJS9_EENSt15__invoke_resultIT_JDpT0_EE4typeEOSK_DpOSL
@ 0x5565e238d62c ZSt6invokeIRSt5_BindIFMN5boost6fibers14worker_contextIZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_JEEEFNS1_7context5fiberEOS9_EPS7_St12_PlaceholderILi1EEEEJS9_EENSt13invoke_resultIT_JDpT0_EE4typeEOSK_DpOSL
*** SIGABRT received at time=1649331134 on cpu 3 ***
PC: @ 0x7fcfd9c7c03b (unknown) raise
@ 0x5565e25896f2 64 absl::lts_20211102::WriteFailureInfo()
@ 0x5565e25898f7 96 absl::lts_20211102::AbslFailureSignalHandler()
@ 0x7fcfd9e3f3c0 (unknown) (unknown)
@ 0x5565e2540ddf 16 google::LogMessage::Fail()
@ 0x5565e2540d21 144 google::LogMessage::SendToLog()
@ 0x5565e2540605 80 google::LogMessage::Flush()
@ 0x5565e2544164 32 google::LogMessageFatal::~LogMessageFatal()
@ 0x5565e1bece27 256 google::CheckNotNull<>()
@ 0x5565e1bbe131 336 dfly::Transaction::ShardArgsInShard()
@ 0x5565e1b62c8b 1008 dfly::StringFamily::MGet()
@ 0x5565e1a2fc0e 80 fu2::abi_400::detail::invocation::invoke<>()
@ 0x5565e1a2aeba 288 fu2::abi_400::detail::type_erasure::invocation_table::function_trait<>::internal_invoker<>::invoke()
@ 0x5565e19c4110 128 fu2::abi_400::detail::type_erasure::tables::vtable<>::invoke<>()
@ 0x5565e19c445c 240 fu2::abi_400::detail::type_erasure::erasure<>::invoke<>()
@ 0x5565e19c4679 240 fu2::abi_400::detail::type_erasure::invocation_table::operator_impl<>::operator()()
@ 0x5565e19b1056 192 dfly::CommandId::Invoke()
@ 0x5565e1947c4d 2336 dfly::Service::DispatchCommand()
@ 0x5565e1f89874 608 facade::Connection::ParseRedis()
@ 0x5565e1f8cf5c 528 facade::Connection::IoLoop()
@ 0x5565e1f86c49 656 facade::Connection::ConnectionFlow()
@ 0x5565e1f8396c 1088 facade::Connection::HandleRequests()
@ 0x5565e236c46c 416 util::ListenerInterface::RunSingleConnection()
@ 0x5565e2366bbb 48 util::ListenerInterface::RunAcceptLoop()::{lambda()#2}::operator()()
@ 0x5565e2384e02 48 std::__invoke_impl<>()
@ 0x5565e2381ec7 32 std::__invoke<>()
@ 0x5565e237cc3b 32 std::_apply_impl<>()
@ 0x5565e237ccb2 48 std::apply<>()
@ 0x5565e237cfd1 240 boost::fibers::worker_context<>::run()
@ 0x5565e239a0be 96 std::__invoke_impl<>()
@ 0x5565e2398f80 96 std::__invoke<>()
@ 0x5565e2398469 224 std::_Bind<>::__call<>()
@ 0x5565e2396be5 176 std::_Bind<>::operator()<>()
@ ... and at least 6 more frames
Aborted (core dumped)

another example:
I20220407 15:58:51.485790 221895 main_service.cc:361] Got: [SET,foo1{t},a]
I20220407 15:58:51.486147 221895 main_service.cc:361] Got: [SET,foo2{t},b]
I20220407 15:58:51.486474 221895 main_service.cc:361] Got: [SET,foo3{t},c]
I20220407 15:58:51.486795 221895 main_service.cc:361] Got: [DEL,foo1{t},foo2{t},foo3{t},foo4{t}]
I20220407 15:58:51.487362 221895 main_service.cc:361] Got: [MGET,foo1{t},foo2{t},foo3{t}]
F20220407 15:58:51.487926 221895 transaction.cc:978] Check failed: 'EngineShard::tlocal()' Must be non NULL
*** Check failure stack trace: ***
@ 0x55ba82b5eddf google::LogMessage::Fail()
@ 0x55ba82b5ed21 google::LogMessage::SendToLog()
@ 0x55ba82b5e605 google::LogMessage::Flush()
@ 0x55ba82b62164 google::LogMessageFatal::~LogMessageFatal()
@ 0x55ba8220ae27 google::CheckNotNull<>()
@ 0x55ba821dc131 dfly::Transaction::ShardArgsInShard()
@ 0x55ba82180c8b dfly::StringFamily::MGet()
@ 0x55ba8204dc0e ZN3fu27abi_4006detail10invocation6invokeIRKPFvN4absl12lts_202111024SpanINS6_IcEEEEPN4dfly17ConnectionContextEEJS8_SB_EEEDTclcl7forwardIT_Efp_Espcl7forwardIT0_Efp0_EEEOSG_DpOSH
@ 0x55ba82048eba fu2::abi_400::detail::type_erasure::invocation_table::function_trait<>::internal_invoker<>::invoke()
@ 0x55ba81fe2110 ZNK3fu27abi_4006detail12type_erasure6tables6vtableINS1_8propertyILb0ELb0EJKFvN4absl12lts_202111024SpanINS8_IcEEEEPN4dfly17ConnectionContextEEEEEE6invokeILm0EJPKNS2_13data_accessorERKmSA_SD_EEEDcDpOT0
@ 0x55ba81fe245c ZN3fu27abi_4006detail12type_erasure7erasureILb1ENS1_6configILb1ELb1ENS_16capacity_defaultEEENS1_8propertyILb0ELb0EJKFvN4absl12lts_202111024SpanINSA_IcEEEEPN4dfly17ConnectionContextEEEEEE6invokeILm0ERKSI_JSC_SF_EEEDcOT0_DpOT1
@ 0x55ba81fe2679 fu2::abi_400::detail::type_erasure::invocation_table::operator_impl<>::operator()()
@ 0x55ba81fcf056 dfly::CommandId::Invoke()
@ 0x55ba81f65c4d dfly::Service::DispatchCommand()
@ 0x55ba825a7874 facade::Connection::ParseRedis()
@ 0x55ba825aaf5c facade::Connection::IoLoop()
@ 0x55ba825a4c49 facade::Connection::ConnectionFlow()
@ 0x55ba825a196c facade::Connection::HandleRequests()
@ 0x55ba8298a46c util::ListenerInterface::RunSingleConnection()
@ 0x55ba82984bbb _ZZN4util17ListenerInterface13RunAcceptLoopEvENKUlvE0_clEv
@ 0x55ba829a2e02 ZSt13__invoke_implIvZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_JEET_St14__invoke_otherOT0_DpOT1
@ 0x55ba8299fec7 ZSt8__invokeIZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_JEENSt15__invoke_resultIT_JDpT0_EE4typeEOS4_DpOS5
@ 0x55ba8299ac3b _ZSt12__apply_implIZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_St5tupleIJEEJEEDcOT_OT0_St16integer_sequenceImJXspT1_EEE
@ 0x55ba8299acb2 ZSt5applyIZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_St5tupleIJEEEDcOT_OT0
@ 0x55ba8299afd1 _ZN5boost6fibers14worker_contextIZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_JEE4run_EONS_7context5fiberE
@ 0x55ba829b80be ZSt13__invoke_implIN5boost7context5fiberERMNS0_6fibers14worker_contextIZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_JEEEFS2_OS2_ERPS8_JS2_EET_St21__invoke_memfun_derefOT0_OT1_DpOT2
@ 0x55ba829b6f80 ZSt8__invokeIRMN5boost6fibers14worker_contextIZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_JEEEFNS0_7context5fiberEOS8_EJRPS6_S8_EENSt15__invoke_resultIT_JDpT0_EE4typeEOSG_DpOSH
@ 0x55ba829b6469 _ZNSt5_BindIFMN5boost6fibers14worker_contextIZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_JEEEFNS0_7context5fiberEOS8_EPS6_St12_PlaceholderILi1EEEE6__callIS8_JS9_EJLm0ELm1EEEET_OSt5tupleIJDpT0_EESt12_Index_tupleIJXspT1_EEE
@ 0x55ba829b4be5 ZNSt5_BindIFMN5boost6fibers14worker_contextIZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_JEEEFNS0_7context5fiberEOS8_EPS6_St12_PlaceholderILi1EEEEclIJS8_ES8_EET0_DpOT
@ 0x55ba829b1d93 ZSt13__invoke_implIN5boost7context5fiberERSt5_BindIFMNS0_6fibers14worker_contextIZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_JEEEFS2_OS2_EPS9_St12_PlaceholderILi1EEEEJS2_EET_St14__invoke_otherOT0_DpOT1
@ 0x55ba829af979 ZSt8__invokeIRSt5_BindIFMN5boost6fibers14worker_contextIZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_JEEEFNS1_7context5fiberEOS9_EPS7_St12_PlaceholderILi1EEEEJS9_EENSt15__invoke_resultIT_JDpT0_EE4typeEOSK_DpOSL
@ 0x55ba829ab62c ZSt6invokeIRSt5_BindIFMN5boost6fibers14worker_contextIZN4util17ListenerInterface13RunAcceptLoopEvEUlvE0_JEEEFNS1_7context5fiberEOS9_EPS7_St12_PlaceholderILi1EEEEJS9_EENSt13invoke_resultIT_JDpT0_EE4typeEOSK_DpOSL
*** SIGABRT received at time=1649336331 on cpu 3 ***
PC: @ 0x7f37e827403b (unknown) raise
@ 0x55ba82ba76f2 64 absl::lts_20211102::WriteFailureInfo()
@ 0x55ba82ba78f7 96 absl::lts_20211102::AbslFailureSignalHandler()
@ 0x7f37e84373c0 (unknown) (unknown)
@ 0x55ba82b5eddf 16 google::LogMessage::Fail()
@ 0x55ba82b5ed21 144 google::LogMessage::SendToLog()
@ 0x55ba82b5e605 80 google::LogMessage::Flush()
@ 0x55ba82b62164 32 google::LogMessageFatal::~LogMessageFatal()
@ 0x55ba8220ae27 256 google::CheckNotNull<>()
@ 0x55ba821dc131 336 dfly::Transaction::ShardArgsInShard()
@ 0x55ba82180c8b 1008 dfly::StringFamily::MGet()
@ 0x55ba8204dc0e 80 fu2::abi_400::detail::invocation::invoke<>()
@ 0x55ba82048eba 288 fu2::abi_400::detail::type_erasure::invocation_table::function_trait<>::internal_invoker<>::invoke()
@ 0x55ba81fe2110 128 fu2::abi_400::detail::type_erasure::tables::vtable<>::invoke<>()
@ 0x55ba81fe245c 240 fu2::abi_400::detail::type_erasure::erasure<>::invoke<>()
@ 0x55ba81fe2679 240 fu2::abi_400::detail::type_erasure::invocation_table::operator_impl<>::operator()()
@ 0x55ba81fcf056 192 dfly::CommandId::Invoke()
@ 0x55ba81f65c4d 2336 dfly::Service::DispatchCommand()
@ 0x55ba825a7874 608 facade::Connection::ParseRedis()
@ 0x55ba825aaf5c 528 facade::Connection::IoLoop()
@ 0x55ba825a4c49 656 facade::Connection::ConnectionFlow()
@ 0x55ba825a196c 1088 facade::Connection::HandleRequests()
@ 0x55ba8298a46c 416 util::ListenerInterface::RunSingleConnection()
@ 0x55ba82984bbb 48 util::ListenerInterface::RunAcceptLoop()::{lambda()#2}::operator()()
@ 0x55ba829a2e02 48 std::__invoke_impl<>()
@ 0x55ba8299fec7 32 std::__invoke<>()
@ 0x55ba8299ac3b 32 std::_apply_impl<>()
@ 0x55ba8299acb2 48 std::apply<>()
@ 0x55ba8299afd1 240 boost::fibers::worker_context<>::run()
@ 0x55ba829b80be 96 std::__invoke_impl<>()
@ 0x55ba829b6f80 96 std::__invoke<>()
@ 0x55ba829b6469 224 std::_Bind<>::__call<>()
@ 0x55ba829b4be5 176 std::_Bind<>::operator()<>()
@ ... and at least 6 more frames
Aborted (core dumped)

This time maybe it can be reproduced by running the following 4 times:
./runtest --host localhost --port 6379 --tags -cluster:skip --ignore-encoding --ignore-digest --quiet --singledb --single unit/keyspace --tags -needs:debug --skiptest "KEYS"

flushdb/flushall and blpop

REDIS:
127.0.0.1:6380> zremrangebyscore fooz 1 NaN
(error) ERR min or max is not a float

DRAGONFLY:
127.0.0.1:6379> zremrangebyscore fooz 1 NaN
(integer) 0

run this (I wrote this twice on purpose):
./runtest --host localhost --port 6379 --tags -cluster:skip --ignore-encoding --ignore-digest --quiet --singledb --single unit/type/list --skiptest "check compression" --skiptest "LPOS|LMOV|BLMPOP|BRPOPLPUSH|RPOPLPUSH|LMPOP|CLIENT|STORE" --tags -needs:debug --skiptest "client unblock tests" --skiptest "List ziplist of various encodings"

./runtest --host localhost --port 6379 --tags -cluster:skip --ignore-encoding --ignore-digest --quiet --singledb --single unit/type/list --skiptest "check compression" --skiptest "LPOS|LMOV|BLMPOP|BRPOPLPUSH|RPOPLPUSH|LMPOP|CLIENT|STORE" --tags -needs:debug --skiptest "client unblock tests" --skiptest "List ziplist of various encodings"

and the result would be:

I20220425 12:34:33.987037 63776 main_service.cc:361] Got: [BLPOP,blist1{t},blist2{t},1]
/home/oded/stratto/dragonfly/src/server/engine_shard_set.cc:274:30: runtime error: member call on misaligned address 0xbebebebebebebebe for type 'struct Transaction', which requires 8 byte alignment
0xbebebebebebebebe: note: pointer points here

/home/oded/stratto/dragonfly/src/server/transaction.cc:1138:25: runtime error: member call on misaligned address 0xbebebebebebebebe for type 'struct Transaction', which requires 8 byte alignment
0xbebebebebebebebe: note: pointer points here

/home/oded/stratto/dragonfly/src/server/transaction.h:202:34: runtime error: member access within misaligned address 0xbebebebebebebebe for type 'const struct Transaction', which requires 8 byte alignment
0xbebebebebebebebe: note: pointer points here

/home/oded/stratto/dragonfly/src/server/transaction.h:202:34: runtime error: member call on misaligned address 0xbebebebebebebefe for type 'const struct InlinedVector', which requires 8 byte alignment
0xbebebebebebebefe: note: pointer points here

/home/oded/stratto/dragonfly/src/server/transaction.h:202:18: runtime error: member access within misaligned address 0xbebebebebebebebe for type 'const struct Transaction', which requires 8 byte alignment
0xbebebebebebebebe: note: pointer points here

/home/oded/stratto/dragonfly/build-dbg/_deps/abseil_cpp-src/absl/container/inlined_vector.h:271:61: runtime error: member access within misaligned address 0xbebebebebebebefe for type 'const struct InlinedVector', which requires 8 byte alignment
0xbebebebebebebefe: note: pointer points here

/home/oded/stratto/dragonfly/build-dbg/_deps/abseil_cpp-src/absl/container/inlined_vector.h:271:61: runtime error: member call on misaligned address 0xbebebebebebebefe for type 'const struct Storage', which requires 8 byte alignment
0xbebebebebebebefe: note: pointer points here

/home/oded/stratto/dragonfly/build-dbg/_deps/abseil_cpp-src/absl/container/inlined_vector.h:271:44: runtime error: member access within misaligned address 0xbebebebebebebefe for type 'const struct InlinedVector', which requires 8 byte alignment
0xbebebebebebebefe: note: pointer points here

/home/oded/stratto/dragonfly/build-dbg/_deps/abseil_cpp-src/absl/container/internal/inlined_vector.h:326:64: runtime error: member call on misaligned address 0xbebebebebebebefe for type 'const struct Storage', which requires 8 byte alignment
0xbebebebebebebefe: note: pointer points here

/home/oded/stratto/dragonfly/build-dbg/_deps/abseil_cpp-src/absl/container/internal/inlined_vector.h:323:38: runtime error: member access within misaligned address 0xbebebebebebebefe for type 'const struct Storage', which requires 8 byte alignment
0xbebebebebebebefe: note: pointer points here

/home/oded/stratto/dragonfly/build-dbg/_deps/abseil_cpp-src/absl/container/internal/inlined_vector.h:323:38: runtime error: member call on misaligned address 0xbebebebebebebefe for type 'const struct Metadata', which requires 8 byte alignment
0xbebebebebebebefe: note: pointer points here

/home/oded/stratto/dragonfly/build-dbg/_deps/abseil_cpp-src/absl/container/internal/inlined_vector.h:323:12: runtime error: member access within misaligned address 0xbebebebebebebefe for type 'const struct Storage', which requires 8 byte alignment
0xbebebebebebebefe: note: pointer points here

/home/oded/stratto/dragonfly/build-dbg/_deps/abseil_cpp-src/absl/container/internal/compressed_tuple.h:265:28: runtime error: member access within misaligned address 0xbebebebebebebefe for type 'const struct CompressedTuple', which requires 8 byte alignment
0xbebebebebebebefe: note: pointer points here

/home/oded/stratto/dragonfly/build-dbg/_deps/abseil_cpp-src/absl/container/internal/compressed_tuple.h:265:28: runtime error: member call on misaligned address 0xbebebebebebebefe for type 'const struct Storage', which requires 8 byte alignment
0xbebebebebebebefe: note: pointer points here

/home/oded/stratto/dragonfly/build-dbg/_deps/abseil_cpp-src/absl/container/internal/compressed_tuple.h:265:28: runtime error: member access within misaligned address 0xbebebebebebebefe for type 'const struct CompressedTuple', which requires 8 byte alignment
0xbebebebebebebefe: note: pointer points here

/home/oded/stratto/dragonfly/build-dbg/_deps/abseil_cpp-src/absl/container/internal/compressed_tuple.h:109:44: runtime error: member access within misaligned address 0xbebebebebebebefe for type 'const struct Storage', which requires 8 byte alignment
0xbebebebebebebefe: note: pointer points here

/home/oded/stratto/dragonfly/build-dbg/_deps/abseil_cpp-src/absl/container/internal/compressed_tuple.h:109:44: runtime error: reference binding to misaligned address 0xbebebebebebebefe for type 'const long unsigned int', which requires 8 byte alignment
0xbebebebebebebefe: note: pointer points here

/home/oded/stratto/dragonfly/build-dbg/_deps/abseil_cpp-src/absl/container/internal/compressed_tuple.h:109:44: runtime error: member access within misaligned address 0xbebebebebebebefe for type 'const struct Storage', which requires 8 byte alignment
0xbebebebebebebefe: note: pointer points here

/home/oded/stratto/dragonfly/build-dbg/_deps/abseil_cpp-src/absl/container/internal/compressed_tuple.h:265:29: runtime error: reference binding to misaligned address 0xbebebebebebebefe for type 'const ElemT', which requires 8 byte alignment
0xbebebebebebebefe: note: pointer points here

/home/oded/stratto/dragonfly/build-dbg/_deps/abseil_cpp-src/absl/container/internal/inlined_vector.h:323:38: runtime error: reference binding to misaligned address 0xbebebebebebebefe for type 'const SizeType', which requires 8 byte alignment
0xbebebebebebebefe: note: pointer points here

/home/oded/stratto/dragonfly/build-dbg/_deps/abseil_cpp-src/absl/container/internal/inlined_vector.h:326:64: runtime error: load of misaligned address 0xbebebebebebebefe for type 'const SizeType', which requires 8 byte alignment
0xbebebebebebebefe: note: pointer points here

*** SIGSEGV received at time=1650879273 on cpu 1 ***
PC: @ 0x563ccd660db7 (unknown) absl::lts_20211102::inlined_vector_internal::Storage<>::GetSize()
@ 0x563cce12e942 64 absl::lts_20211102::WriteFailureInfo()
@ 0x563cce12eb47 96 absl::lts_20211102::AbslFailureSignalHandler()
@ 0x7feac3d003c0 (unknown) (unknown)
@ 0x563ccd65cc60 32 absl::lts_20211102::InlinedVector<>::size()
@ 0x563ccd6f61b7 48 dfly::Transaction::SidToId()
@ 0x563ccd6d744f 528 dfly::Transaction::NotifySuspended()
@ 0x563ccd8853d1 80 dfly::EngineShard::NotifyWatchQueue()
@ 0x563ccd886875 1360 dfly::EngineShard::ProcessAwakened()
@ 0x563ccd6b8425 1216 dfly::Transaction::RunInShard()
@ 0x563ccd88243e 1376 dfly::EngineShard::PollExecution()
@ 0x563ccd6c5d04 448 dfly::Transaction::ExecuteAsync()::{lambda()#1}::operator()()
@ 0x563ccd6ea9f7 48 std::_Function_handler<>::_M_invoke()
@ 0x563ccdb3600b 48 std::function<>::operator()()
@ 0x563ccdc0f948 352 util::fibers_ext::FiberQueue::Run()
@ 0x563ccd87d56d 384 dfly::EngineShard::EngineShard()::{lambda()#2}::operator()()
@ 0x563ccd891cd5 48 std::__invoke_impl<>()
@ 0x563ccd8915de 32 std::__invoke<>()
@ 0x563ccd890d7a 32 std::_apply_impl<>()
@ 0x563ccd890df1 48 std::apply<>()
@ 0x563ccd891111 240 boost::fibers::worker_context<>::run()
@ 0x563ccd894741 96 std::__invoke_impl<>()
@ 0x563ccd894252 96 std::__invoke<>()
@ 0x563ccd893f89 224 std::_Bind<>::__call<>()
@ 0x563ccd893be9 176 std::_Bind<>::operator()<>()
@ 0x563ccd893a2d 80 std::__invoke_impl<>()
@ 0x563ccd8935fb 80 std::__invoke<>()
@ 0x563ccd893410 80 std::invoke<>()
@ 0x563ccd892d7d 256 boost::context::detail::fiber_record<>::run()
@ 0x563ccd892450 224 boost::context::detail::fiber_entry<>()
@ 0x7feac4ab51cf (unknown) make_fcontext
Segmentation fault (core dumped)

zadd myz 1 x
zincrby myz 1 x
zincrby myz nan x
result should be:
(error) value is not a valid float

zadd myz nan x
result should be:
(error) value is not a valid float

currently the result is:
(error) ERR resulting score is not a number (NaN)

DragonflyDB appears to be continuously polling, with no sleeps or wakeups, on all CPUs even when completely idle.

Beyond energy efficiency, this makes it a liability in combination with any laptop battery, or any environment where Dragonfly must run alongside other servers (say, a classic LAMP style VPS environment)

Is there any way to adjust the design so that CPU usage is proportional to load? Ideally there are no wakeups when there are no connections or background work, although this is something Redis never managed to achieve either.

I succeed to reproduce it on instance x2gd.8xlarge, ubuntu 20.04.4 by running:

while true; do memtier_benchmark --ratio 1:0 -n 1000 -d 120000; done

at some point the server gets stuck, and netstat -atl shows a socket with unread data in its recv-q.

Seems like a missed completion from io_uring but I have not confirmed it with 100%.
I did confirm that my Recv callback does not run but it could theoretically be because of some corner case in my code.

Having said that, I did not succeed to reproduce the deadlock on 22.04, so current working theory is that the bug is in the kernel (io
_uring).

./runtest --host localhost --port 6379 --tags -cluster:skip --ignore-encoding --ignore-digest --quiet --singledb --single unit/type/list --skiptest "check compression" --skiptest "LINSERT|LPOS|LMOV|BLMPOP" --skiptest BRPOPLPUSH --tags -needs:debug --tags -large-memory --skiptest "LPOP/RPOP with wrong number of arguments" --skiptest "LPOP/RPOP with the count 0 returns an empty array in RESP2" --skiptest "LPOP/RPOP with against non existing key in RESP2"

I20220406 08:26:48.868800 174880 main_service.cc:361] Got: [INFO]
I20220406 08:26:48.972268 174880 main_service.cc:361] Got: [INFO]
I20220406 08:26:49.076424 174880 main_service.cc:361] Got: [INFO]
I20220406 08:26:49.179415 174880 main_service.cc:361] Got: [INFO]
I20220406 08:26:49.282332 174880 main_service.cc:361] Got: [INFO]
I20220406 08:26:49.385401 174880 main_service.cc:361] Got: [INFO]
I20220406 08:26:49.490737 174879 main_service.cc:361] Got: [DEBUG,log,### Starting test BLPOP with same key multiple times should work (issue #801) in tests/unit/type/list.tcl]
I20220406 08:26:49.493136 174880 main_service.cc:361] Got: [PING]
I20220406 08:26:49.493824 174880 main_service.cc:361] Got: [DEL,list1{t},list2{t}]
I20220406 08:26:49.495383 174880 main_service.cc:361] Got: [BLPOP,list1{t},list2{t},list2{t},list1{t},0]
I20220406 08:26:49.496011 174880 main_service.cc:361] Got: [LPUSH,list1{t},a]
F20220406 08:26:49.497831 174877 transaction.cc:1135] Check failed: 0u != local_mask & SUSPENDED_Q (0 vs. 0)
*** Check failure stack trace: ***
@ 0x557ef3285b2f google::LogMessage::Fail()
@ 0x557ef3285a71 google::LogMessage::SendToLog()
@ 0x557ef3285355 google::LogMessage::Flush()
@ 0x557ef3288eb4 google::LogMessageFatal::~LogMessageFatal()
@ 0x557ef291d4da dfly::Transaction::NotifySuspended()
@ 0x557ef2ab4a20 dfly::EngineShard::ProcessAwakened()
@ 0x557ef28feac9 dfly::Transaction::RunInShard()
@ 0x557ef2aaf029 dfly::EngineShard::PollExecution()
@ 0x557ef290c198 _ZZN4dfly11Transaction12ExecuteAsyncEvENKUlvE_clEv
@ 0x557ef2930983 _ZNSt17_Function_handlerIFvvEZN4dfly11Transaction12ExecuteAsyncEvEUlvE_E9_M_invokeERKSt9_Any_data
@ 0x557ef2ce95b7 std::function<>::operator()()
@ 0x557ef2db9118 util::fibers_ext::FiberQueue::Run()
@ 0x557ef2aab25d _ZZN4dfly11EngineShardC4EPN4util12ProactorBaseEbP9mi_heap_sENKUlvE0_clEv
@ 0x557ef2abe749 ZSt13__invoke_implIvZN4dfly11EngineShardC4EPN4util12ProactorBaseEbP9mi_heap_sEUlvE0_JEET_St14__invoke_otherOT0_DpOT1
@ 0x557ef2abe052 ZSt8__invokeIZN4dfly11EngineShardC4EPN4util12ProactorBaseEbP9mi_heap_sEUlvE0_JEENSt15__invoke_resultIT_JDpT0_EE4typeEOS9_DpOSA
@ 0x557ef2abd7ee _ZSt12__apply_implIZN4dfly11EngineShardC4EPN4util12ProactorBaseEbP9mi_heap_sEUlvE0_St5tupleIJEEJEEDcOT_OT0_St16integer_sequenceImJXspT1_EEE
@ 0x557ef2abd865 ZSt5applyIZN4dfly11EngineShardC4EPN4util12ProactorBaseEbP9mi_heap_sEUlvE0_St5tupleIJEEEDcOT_OT0
@ 0x557ef2abdb85 _ZN5boost6fibers14worker_contextIZN4dfly11EngineShardC4EPN4util12ProactorBaseEbP9mi_heap_sEUlvE0_JEE4run_EONS_7context5fiberE
@ 0x557ef2ac11b5 ZSt13__invoke_implIN5boost7context5fiberERMNS0_6fibers14worker_contextIZN4dfly11EngineShardC4EPN4util12ProactorBaseEbP9mi_heap_sEUlvE0_JEEEFS2_OS2_ERPSD_JS2_EET_St21__invoke_memfun_derefOT0_OT1_DpOT2
@ 0x557ef2ac0cc6 ZSt8__invokeIRMN5boost6fibers14worker_contextIZN4dfly11EngineShardC4EPN4util12ProactorBaseEbP9mi_heap_sEUlvE0_JEEEFNS0_7context5fiberEOSD_EJRPSB_SD_EENSt15__invoke_resultIT_JDpT0_EE4typeEOSL_DpOSM
@ 0x557ef2ac09fd _ZNSt5_BindIFMN5boost6fibers14worker_contextIZN4dfly11EngineShardC4EPN4util12ProactorBaseEbP9mi_heap_sEUlvE0_JEEEFNS0_7context5fiberEOSD_EPSB_St12_PlaceholderILi1EEEE6__callISD_JSE_EJLm0ELm1EEEET_OSt5tupleIJDpT0_EESt12_Index_tupleIJXspT1_EEE
@ 0x557ef2ac065d ZNSt5_BindIFMN5boost6fibers14worker_contextIZN4dfly11EngineShardC4EPN4util12ProactorBaseEbP9mi_heap_sEUlvE0_JEEEFNS0_7context5fiberEOSD_EPSB_St12_PlaceholderILi1EEEEclIJSD_ESD_EET0_DpOT
@ 0x557ef2ac04a1 ZSt13__invoke_implIN5boost7context5fiberERSt5_BindIFMNS0_6fibers14worker_contextIZN4dfly11EngineShardC4EPN4util12ProactorBaseEbP9mi_heap_sEUlvE0_JEEEFS2_OS2_EPSE_St12_PlaceholderILi1EEEEJS2_EET_St14__invoke_otherOT0_DpOT1
@ 0x557ef2ac006f ZSt8__invokeIRSt5_BindIFMN5boost6fibers14worker_contextIZN4dfly11EngineShardC4EPN4util12ProactorBaseEbP9mi_heap_sEUlvE0_JEEEFNS1_7context5fiberEOSE_EPSC_St12_PlaceholderILi1EEEEJSE_EENSt15__invoke_resultIT_JDpT0_EE4typeEOSP_DpOSQ
@ 0x557ef2abfe84 ZSt6invokeIRSt5_BindIFMN5boost6fibers14worker_contextIZN4dfly11EngineShardC4EPN4util12ProactorBaseEbP9mi_heap_sEUlvE0_JEEEFNS1_7context5fiberEOSE_EPSC_St12_PlaceholderILi1EEEEJSE_EENSt13invoke_resultIT_JDpT0_EE4typeEOSP_DpOSQ
@ 0x557ef2abf7f1 _ZN5boost7context6detail12fiber_recordINS0_5fiberENS0_21basic_fixedsize_stackINS0_12stack_traitsEEESt5_BindIFMNS_6fibers14worker_contextIZN4dfly11EngineShardC4EPN4util12ProactorBaseEbP9mi_heap_sEUlvE0_JEEEFS3_OS3_EPSI_St12_PlaceholderILi1EEEEE3runEPv
@ 0x557ef2abeec4 _ZN5boost7context6detail11fiber_entryINS1_12fiber_recordINS0_5fiberENS0_21basic_fixedsize_stackINS0_12stack_traitsEEESt5_BindIFMNS_6fibers14worker_contextIZN4dfly11EngineShardC4EPN4util12ProactorBaseEbP9mi_heap_sEUlvE0_JEEEFS4_OS4_EPSJ_St12_PlaceholderILi1EEEEEEEEvNS1_10transfer_tE
@ 0x7f4fb02e01cf make_fcontext
*** SIGABRT received at time=1649222809 on cpu 0 ***
PC: @ 0x7f4faf36803b (unknown) raise
@ 0x557ef32ce442 64 absl::lts_20211102::WriteFailureInfo()
@ 0x557ef32ce647 96 absl::lts_20211102::AbslFailureSignalHandler()
@ 0x7f4fb01e63c0 (unknown) (unknown)
@ 0x557ef3285b2f 16 google::LogMessage::Fail()
@ 0x557ef3285a71 144 google::LogMessage::SendToLog()
@ 0x557ef3285355 80 google::LogMessage::Flush()
@ 0x557ef3288eb4 32 google::LogMessageFatal::~LogMessageFatal()
@ 0x557ef291d4da 560 dfly::Transaction::NotifySuspended()
@ 0x557ef2ab4a20 1296 dfly::EngineShard::ProcessAwakened()
@ 0x557ef28feac9 1216 dfly::Transaction::RunInShard()
@ 0x557ef2aaf029 1376 dfly::EngineShard::PollExecution()
@ 0x557ef290c198 448 dfly::Transaction::ExecuteAsync()::{lambda()#1}::operator()()
@ 0x557ef2930983 48 std::_Function_handler<>::_M_invoke()
@ 0x557ef2ce95b7 48 std::function<>::operator()()
@ 0x557ef2db9118 352 util::fibers_ext::FiberQueue::Run()
@ 0x557ef2aab25d 384 dfly::EngineShard::EngineShard()::{lambda()#2}::operator()()
@ 0x557ef2abe749 48 std::__invoke_impl<>()
@ 0x557ef2abe052 32 std::__invoke<>()
@ 0x557ef2abd7ee 32 std::_apply_impl<>()
@ 0x557ef2abd865 48 std::apply<>()
@ 0x557ef2abdb85 240 boost::fibers::worker_context<>::run()
@ 0x557ef2ac11b5 96 std::__invoke_impl<>()
@ 0x557ef2ac0cc6 96 std::__invoke<>()
@ 0x557ef2ac09fd 224 std::_Bind<>::__call<>()
@ 0x557ef2ac065d 176 std::_Bind<>::operator()<>()
@ 0x557ef2ac04a1 80 std::__invoke_impl<>()
@ 0x557ef2ac006f 80 std::__invoke<>()
@ 0x557ef2abfe84 80 std::invoke<>()
@ 0x557ef2abf7f1 256 boost::context::detail::fiber_record<>::run()
@ 0x557ef2abeec4 224 boost::context::detail::fiber_entry<>()
@ 0x7f4fb02e01cf (unknown) make_fcontext
Aborted (core dumped)

Another example that looks related:

./runtest --host localhost --port 6379 --tags -cluster:skip --ignore-encoding --ignore-digest --quiet --singledb --single unit/type/list --skiptest "check compression" --skiptest "LINSERT|LPOS|LMOV|BLMPOP" --skiptest BRPOPLPUSH --tags -needs:debug --tags -large-memory --skiptest "LPOP/RPOP with wrong number of arguments" --skiptest "LPOP/RPOP with the count 0 returns an empty array in RESP2" --skiptest "LPOP/RPOP with against non existing key in RESP2" --skiptest "BLPOP with same key multiple times should work"

I20220406 08:37:29.740819 175232 main_service.cc:361] Got: [INFO]
I20220406 08:37:29.842866 175232 main_service.cc:361] Got: [INFO]
I20220406 08:37:29.944725 175232 main_service.cc:361] Got: [INFO]
I20220406 08:37:30.050365 175232 main_service.cc:361] Got: [INFO]
I20220406 08:37:30.153288 175232 main_service.cc:361] Got: [INFO]
I20220406 08:37:30.255180 175232 main_service.cc:361] Got: [INFO]
I20220406 08:37:30.358577 175229 main_service.cc:361] Got: [DEBUG,log,### Starting test BRPOP: with negative timeout in tests/unit/type/list.tcl]
I20220406 08:37:30.359725 175230 main_service.cc:361] Got: [PING]
I20220406 08:37:30.360023 175230 main_service.cc:361] Got: [BRPOP,blist1,-1]
I20220406 08:37:30.361078 175231 main_service.cc:361] Got: [DEBUG,log,### Starting test BRPOP: with non-integer timeout in tests/unit/type/list.tcl]
I20220406 08:37:30.362107 175232 main_service.cc:361] Got: [PING]
I20220406 08:37:30.362403 175232 main_service.cc:361] Got: [DEL,blist1]
I20220406 08:37:30.362874 175232 main_service.cc:361] Got: [BRPOP,blist1,0.1]
I20220406 08:37:30.363121 175232 main_service.cc:361] Got: [RPUSH,blist1,foo]
F20220406 08:37:30.364477 175231 transaction.cc:1143] Check failed: 0u == local_mask & AWAKED_Q (0 vs. 32)
*** Check failure stack trace: ***
@ 0x557b9137bb2f google::LogMessage::Fail()
@ 0x557b9137ba71 google::LogMessage::SendToLog()
@ 0x557b9137b355 google::LogMessage::Flush()
@ 0x557b9137eeb4 google::LogMessageFatal::~LogMessageFatal()
@ 0x557b90a13b5f dfly::Transaction::NotifySuspended()
@ 0x557b90baaa20 dfly::EngineShard::ProcessAwakened()
@ 0x557b909f4ac9 dfly::Transaction::RunInShard()
@ 0x557b90ba5029 dfly::EngineShard::PollExecution()
@ 0x557b90a02198 _ZZN4dfly11Transaction12ExecuteAsyncEvENKUlvE_clEv
@ 0x557b90a26983 _ZNSt17_Function_handlerIFvvEZN4dfly11Transaction12ExecuteAsyncEvEUlvE_E9_M_invokeERKSt9_Any_data
@ 0x557b90ddf5b7 std::function<>::operator()()
@ 0x557b90eaf118 util::fibers_ext::FiberQueue::Run()
@ 0x557b90ba125d _ZZN4dfly11EngineShardC4EPN4util12ProactorBaseEbP9mi_heap_sENKUlvE0_clEv
@ 0x557b90bb4749 ZSt13__invoke_implIvZN4dfly11EngineShardC4EPN4util12ProactorBaseEbP9mi_heap_sEUlvE0_JEET_St14__invoke_otherOT0_DpOT1
@ 0x557b90bb4052 ZSt8__invokeIZN4dfly11EngineShardC4EPN4util12ProactorBaseEbP9mi_heap_sEUlvE0_JEENSt15__invoke_resultIT_JDpT0_EE4typeEOS9_DpOSA
@ 0x557b90bb37ee _ZSt12__apply_implIZN4dfly11EngineShardC4EPN4util12ProactorBaseEbP9mi_heap_sEUlvE0_St5tupleIJEEJEEDcOT_OT0_St16integer_sequenceImJXspT1_EEE
@ 0x557b90bb3865 ZSt5applyIZN4dfly11EngineShardC4EPN4util12ProactorBaseEbP9mi_heap_sEUlvE0_St5tupleIJEEEDcOT_OT0
@ 0x557b90bb3b85 _ZN5boost6fibers14worker_contextIZN4dfly11EngineShardC4EPN4util12ProactorBaseEbP9mi_heap_sEUlvE0_JEE4run_EONS_7context5fiberE
@ 0x557b90bb71b5 ZSt13__invoke_implIN5boost7context5fiberERMNS0_6fibers14worker_contextIZN4dfly11EngineShardC4EPN4util12ProactorBaseEbP9mi_heap_sEUlvE0_JEEEFS2_OS2_ERPSD_JS2_EET_St21__invoke_memfun_derefOT0_OT1_DpOT2
@ 0x557b90bb6cc6 ZSt8__invokeIRMN5boost6fibers14worker_contextIZN4dfly11EngineShardC4EPN4util12ProactorBaseEbP9mi_heap_sEUlvE0_JEEEFNS0_7context5fiberEOSD_EJRPSB_SD_EENSt15__invoke_resultIT_JDpT0_EE4typeEOSL_DpOSM
@ 0x557b90bb69fd _ZNSt5_BindIFMN5boost6fibers14worker_contextIZN4dfly11EngineShardC4EPN4util12ProactorBaseEbP9mi_heap_sEUlvE0_JEEEFNS0_7context5fiberEOSD_EPSB_St12_PlaceholderILi1EEEE6__callISD_JSE_EJLm0ELm1EEEET_OSt5tupleIJDpT0_EESt12_Index_tupleIJXspT1_EEE
@ 0x557b90bb665d ZNSt5_BindIFMN5boost6fibers14worker_contextIZN4dfly11EngineShardC4EPN4util12ProactorBaseEbP9mi_heap_sEUlvE0_JEEEFNS0_7context5fiberEOSD_EPSB_St12_PlaceholderILi1EEEEclIJSD_ESD_EET0_DpOT
@ 0x557b90bb64a1 ZSt13__invoke_implIN5boost7context5fiberERSt5_BindIFMNS0_6fibers14worker_contextIZN4dfly11EngineShardC4EPN4util12ProactorBaseEbP9mi_heap_sEUlvE0_JEEEFS2_OS2_EPSE_St12_PlaceholderILi1EEEEJS2_EET_St14__invoke_otherOT0_DpOT1
@ 0x557b90bb606f ZSt8__invokeIRSt5_BindIFMN5boost6fibers14worker_contextIZN4dfly11EngineShardC4EPN4util12ProactorBaseEbP9mi_heap_sEUlvE0_JEEEFNS1_7context5fiberEOSE_EPSC_St12_PlaceholderILi1EEEEJSE_EENSt15__invoke_resultIT_JDpT0_EE4typeEOSP_DpOSQ
@ 0x557b90bb5e84 ZSt6invokeIRSt5_BindIFMN5boost6fibers14worker_contextIZN4dfly11EngineShardC4EPN4util12ProactorBaseEbP9mi_heap_sEUlvE0_JEEEFNS1_7context5fiberEOSE_EPSC_St12_PlaceholderILi1EEEEJSE_EENSt13invoke_resultIT_JDpT0_EE4typeEOSP_DpOSQ
@ 0x557b90bb57f1 _ZN5boost7context6detail12fiber_recordINS0_5fiberENS0_21basic_fixedsize_stackINS0_12stack_traitsEEESt5_BindIFMNS_6fibers14worker_contextIZN4dfly11EngineShardC4EPN4util12ProactorBaseEbP9mi_heap_sEUlvE0_JEEEFS3_OS3_EPSI_St12_PlaceholderILi1EEEEE3runEPv
@ 0x557b90bb4ec4 _ZN5boost7context6detail11fiber_entryINS1_12fiber_recordINS0_5fiberENS0_21basic_fixedsize_stackINS0_12stack_traitsEEESt5_BindIFMNS_6fibers14worker_contextIZN4dfly11EngineShardC4EPN4util12ProactorBaseEbP9mi_heap_sEUlvE0_JEEEFS4_OS4_EPSJ_St12_PlaceholderILi1EEEEEEEEvNS1_10transfer_tE
@ 0x7f3323ccc1cf make_fcontext
*** SIGABRT received at time=1649223450 on cpu 2 ***
PC: @ 0x7f3322d5403b (unknown) raise
@ 0x557b913c4442 64 absl::lts_20211102::WriteFailureInfo()
@ 0x557b913c4647 96 absl::lts_20211102::AbslFailureSignalHandler()
@ 0x7f3323bd23c0 (unknown) (unknown)
@ 0x557b9137bb2f 16 google::LogMessage::Fail()
@ 0x557b9137ba71 144 google::LogMessage::SendToLog()
@ 0x557b9137b355 80 google::LogMessage::Flush()
@ 0x557b9137eeb4 32 google::LogMessageFatal::~LogMessageFatal()
@ 0x557b90a13b5f 560 dfly::Transaction::NotifySuspended()
@ 0x557b90baaa20 1296 dfly::EngineShard::ProcessAwakened()
@ 0x557b909f4ac9 1216 dfly::Transaction::RunInShard()
@ 0x557b90ba5029 1376 dfly::EngineShard::PollExecution()
@ 0x557b90a02198 448 dfly::Transaction::ExecuteAsync()::{lambda()#1}::operator()()
@ 0x557b90a26983 48 std::_Function_handler<>::_M_invoke()
@ 0x557b90ddf5b7 48 std::function<>::operator()()
@ 0x557b90eaf118 352 util::fibers_ext::FiberQueue::Run()
@ 0x557b90ba125d 384 dfly::EngineShard::EngineShard()::{lambda()#2}::operator()()
@ 0x557b90bb4749 48 std::__invoke_impl<>()
@ 0x557b90bb4052 32 std::__invoke<>()
@ 0x557b90bb37ee 32 std::_apply_impl<>()
@ 0x557b90bb3865 48 std::apply<>()
@ 0x557b90bb3b85 240 boost::fibers::worker_context<>::run()
@ 0x557b90bb71b5 96 std::__invoke_impl<>()
@ 0x557b90bb6cc6 96 std::__invoke<>()
@ 0x557b90bb69fd 224 std::_Bind<>::__call<>()
@ 0x557b90bb665d 176 std::_Bind<>::operator()<>()
@ 0x557b90bb64a1 80 std::__invoke_impl<>()
@ 0x557b90bb606f 80 std::__invoke<>()
@ 0x557b90bb5e84 80 std::invoke<>()
@ 0x557b90bb57f1 256 boost::context::detail::fiber_record<>::run()
@ 0x557b90bb4ec4 224 boost::context::detail::fiber_entry<>()
@ 0x7f3323ccc1cf (unknown) make_fcontext
Aborted (core dumped)

docker run --network=host --rm dragonfly -alsologtostderr

./runtest --host localhost --port 6379 --tags -cluster:skip --ignore-encoding --ignore-digest --quiet --singledb --single unit/type/list --skiptest "check compression" --skiptest "LPOS|LMOV|BLMPOP|BRPOPLPUSH|RPOPLPUSH|LMPOP|CLIENT|STORE" --tags -needs:debug --skiptest "client unblock tests" --skiptest "List ziplist of various encodings"

./runtest --host localhost --port 6379 --tags -cluster:skip --ignore-encoding --ignore-digest --quiet --singledb --single unit/type/list --skiptest "check compression" --skiptest "LPOS|LMOV|BLMPOP|BRPOPLPUSH|RPOPLPUSH|LMPOP|CLIENT|STORE" --tags -needs:debug --skiptest "client unblock tests" --skiptest "List ziplist of various encodings"

./runtest --host localhost --port 6379 --tags -cluster:skip --ignore-encoding --ignore-digest --quiet --singledb --single unit/type/list --skiptest "check compression" --skiptest "LPOS|LMOV|BLMPOP|BRPOPLPUSH|RPOPLPUSH|LMPOP|CLIENT|STORE" --tags -needs:debug --skiptest "client unblock tests" --skiptest "List ziplist of various encodings"

first run pass
second time pass
3rd time crush the server with the following:

I20220526 09:36:18.267655 1 init.cc:55] dragonfly running in opt mode.
I20220526 09:36:18.267711 1 dfly_main.cc:100] maxmemory has not been specified. Deciding myself....
I20220526 09:36:18.267808 1 dfly_main.cc:105] Found 4.31GiB available memory. Setting maxmemory to 3.45GiB
I20220526 09:36:18.268343 11 proactor.cc:415] IORing with 1024 entries, allocated 102720 bytes, cq_entries is 2048
I20220526 09:36:18.269382 1 proactor_pool.cc:65] Running 4 io threads
I20220526 09:36:18.270294 1 server_family.cc:179] Data directory is "/data"
I20220526 09:36:18.270406 1 server_family.cc:107] Checking "/data/dump"
I20220526 09:36:18.270684 12 listener_interface.cc:79] sock[15] AcceptServer - listening on port 6379
F20220526 09:36:33.371167 11 transaction.cc:1163] Check failed: sd.local_mask & AWAKED_Q
*** Check failure stack trace: ***
@ 0x558795426560 google::LogMessage::Fail()
@ 0x558795429621 google::LogMessage::SendToLog()
@ 0x55879542603c google::LogMessage::Flush()
@ 0x558795429d1f google::LogMessageFatal::~LogMessageFatal()
@ 0x55879535e2f0 dfly::BlockingController::RunStep()
@ 0x5587953780c8 dfly::Transaction::RunInShard()
@ 0x5587952b8a69 dfly::EngineShard::PollExecution()
@ 0x558795373545 _ZNSt17_Function_handlerIFvvEZN4dfly11Transaction12ExecuteAsyncEvEUlvE_E9_M_invokeERKSt9_Any_data.lto_priv.0
@ 0x55879533e6a1 _ZN5boost6fibers14worker_contextIZN4dfly11EngineShardC4EPN4util12ProactorBaseEbP9mi_heap_sEUlvE0_JEE4run_EONS_7context5fiberE
@ 0x55879533b673 _ZN5boost7context6detail11fiber_entryINS1_12fiber_recordINS0_5fiberENS0_21basic_fixedsize_stackINS0_12stack_traitsEEESt5_BindIFMNS_6fibers14worker_contextIZN4dfly11EngineShardC4EPN4util12ProactorBaseEbP9mi_heap_sEUlvE0_JEEEFS4_OS4_EPSJ_St12_PlaceholderILi1EEEEEEEEvNS1_10transfer_tE
@ 0x5587954027df make_fcontext
*** SIGABRT received at time=1653557793 on cpu 1 ***
PC: @ 0x7fa0dba5403b (unknown) raise
@ ... and at least 1 more frames
[failure_signal_handler.cc : 331] RAW: Signal 11 raised at PC=0x7fa0dba33941 while already in AbslFailureSignalHandler()
*** SIGSEGV received at time=1653557793 on cpu 1 ***
PC: @ 0x7fa0dba33941 (unknown) abort
@ ... and at least 1 more frames

    DEL zset

ZADD zset -inf a
ZADD zset 1 b
ZADD zset 2 c
ZADD zset 3 d
ZADD zset 4 e
ZADD zset 5 f
ZADD zset +inf g
zrevrangebyscore zset 2 -inf
should return c b a

Fix load on start - right now we do not load a file upon starting the database.

fix save naming scheme. I would like to change the default of overriding the same file again and again.
But we should provide a config option to fallback to redis behavior.

there are many existing libraries for redis in different languages is this a drop-in replacement for redis or are you planning to develop these?

nvm it is

I20220425 13:09:32.508553 65285 main_service.cc:361] Got: [ZREVRANGE,ztmp,0,-1,withscores]
ERR wrong number of arguments for 'ZREVRANGE' command.

oded@OdedCmp:~/stratto/redis$ ./runtest --host localhost --port 6379 --tags -cluster:skip --ignore-encoding --ignore-digest --quiet --singledb --single unit/type/set --tags -needs:debug --skiptest "SINTERCARD|SRANDMEMBER|WATCH|SMISMEMBER" --skiptest "SMOVE only notify dstset when the addition is successful" --skiptest "SADD an integer larger than 64 bits"
[err]: SDIFF against non-set should throw error in tests/unit/type/set.tcl
Expected an error matching 'WRONGTYPE*' but got '' (context: type eval line 6 cmd {assert_error "WRONGTYPE*" {r sdiff noset{t} key1{t}}} proc ::test)
[err]: SDIFFSTORE against non-set should throw error in tests/unit/type/set.tcl
Expected an error matching 'WRONGTYPE*' but got '0' (context: type eval line 8 cmd {assert_error "WRONGTYPE*" {r SDIFFSTORE set3{t} noset{t} key1{t}}} proc ::test)
[1/1 done]: unit/type/set (20 seconds)

               The End

Execution time of different units:
20 seconds - unit/type/set

!!! WARNING The following tests failed:

*** [err]: SDIFF against non-set should throw error in tests/unit/type/set.tcl
Expected an error matching 'WRONGTYPE*' but got '' (context: type eval line 6 cmd {assert_error "WRONGTYPE*" {r sdiff noset{t} key1{t}}} proc ::test)
*** [err]: SDIFFSTORE against non-set should throw error in tests/unit/type/set.tcl
Expected an error matching 'WRONGTYPE*' but got '0' (context: type eval line 8 cmd {assert_error "WRONGTYPE*" {r SDIFFSTORE set3{t} noset{t} key1{t}}} proc ::test)

Says '...utra fast in-memory datastore....' instead of '...ultra fast in-memory datastore...'

Currently, each thread updates its expiry clock independently from others and independently from the ongoing transactions.

1. We should move expiration clock sampling to transaction. Transaction should have a constant number used through all its operations,
   otherwise we risk having inconsistent replies within a transaction for GET X...some delay GET X sequences.
2. We should consider having a single atomic variable for the clock so that transactions clocks will have monotonic property. I do not think it's strictly necessary but we may consider it.

del mylist
RPUSH mylist 99
RPUSH mylist 98
RPUSH mylist RomanRoman
RPUSH mylist 96
RPUSH mylist 95
lset mylist 1 foo
lset mylist -1 boo
lset mylist 10 foo

return ERR Internal error should return ERRrange

from list.tcl this is the "LSET out of range index - linkedlist" test but it cannot run individually.

Again, arm64. Specifically x2gd.8xlarge. Reproduced on ubuntu 22.04 and 20.04.

When running dragonfly with 32 threads, its 99 and 99.9 latency percentiles are very high (10-20ms).
To reproduce : memtier_benchmark --ratio 0:1 -n 10000 --hide-histogram

With --command=ping benchmark the latency is adequate so I dug in message-passing code. Indeed I saw round-trip latencies of over 10ms (Say if you measure WaitForShardCallbacks() latency).

Reducing --proactor_threads to 28, 26 helps a lot! 99.9 becomes less than 1ms.
I still do not understand why the latency is so high and whether it's a real bug in dragonfly, problems in kernel or a weird interaction on this specific hardware.

For now we have a workaround. I will need to follow up and see where latency variance becomes so scrazy.

What's the expected docker-compose usage?

and is it possible to use docker port binding?

tested dragonfly --port 6379 --network=host --ulimit memlock=-1 and it doesn't work either