comparing new version of mimalloc 2.1.4 to 2.0.9 shows degradation in unaccounted memory

with 2.0.9
127.0.0.1:6379> DEBUG POPULATE 300000 JSON 100 RAND TYPE JSON ELEMENTS 100
OK
(2.28s)
127.0.0.1:6379> info memory

Memory

used_memory:6679902592
used_memory_human:6.22GiB
used_memory_peak:6679902592
used_memory_peak_human:6.22GiB
fibers_stack_vms:1637600
fibers_count:50
used_memory_rss:6743437312
used_memory_rss_human:6.28GiB
used_memory_peak_rss:6743437312
maxmemory:18635944755
maxmemory_human:17.36GiB
object_used_memory:4800000
type_used_memory_JSON:4800000
table_used_memory:16912128
num_buckets:30720
num_entries:300000
inline_keys:300000
listpack_blobs:0
listpack_bytes:0
small_string_bytes:0
pipeline_cache_bytes:0
dispatch_queue_bytes:0
dispatch_queue_subscriber_bytes:0
dispatch_queue_peak_bytes:0
client_read_buffer_peak_bytes:256
tls_bytes:0
cache_mode:store
maxmemory_policy:noeviction

with 2.1.4
127.0.0.1:6379> DEBUG POPULATE 300000 JSON 100 RAND TYPE JSON ELEMENTS 100
OK
(2.29s)
127.0.0.1:6379> info memory

Memory

used_memory:6366138816
used_memory_human:5.93GiB
used_memory_peak:6366138816
used_memory_peak_human:5.93GiB
fibers_stack_vms:1637600
fibers_count:50
used_memory_rss:7705546752
used_memory_rss_human:7.18GiB
used_memory_peak_rss:7705546752
maxmemory:17508161945
maxmemory_human:16.31GiB
object_used_memory:4800000
type_used_memory_JSON:4800000
table_used_memory:16912128
num_buckets:30720
num_entries:300000
inline_keys:300000
listpack_blobs:0
listpack_bytes:0
small_string_bytes:0
pipeline_cache_bytes:0
dispatch_queue_bytes:0
dispatch_queue_subscriber_bytes:0
dispatch_queue_peak_bytes:0
client_read_buffer_peak_bytes:256
tls_bytes:0
cache_mode:store
maxmemory_policy:noeviction

from dragonfly.

i suggest checking memory malloc-stats as well

from dragonfly.

microsoft/mimalloc#887

from dragonfly.

the issue with bigger block allocation, 18488 160 65120 65120 65120 - old vs
22132 184 65280 65280 62560 - new

from dragonfly.

One issue I found we have problems with mi_heap_malloc_aligned.

1. 8 byte alignment should be a noop but in practice it can increase the effective size.
2. with (size % alignment) == 0 we may also have the unjustifiable overhead.

This is not related to theversion update, but was discovered when checking for differences in used memory

from dragonfly.