pelikan-io / pelikan Goto Github PK

Pelikan is a framework for building local or distributed caches. It comes with a highly extensible architecture, best-in-class performance, and superb operational ergonomics. You can use it to replace most of Memcached or a subset of Redis features.

Home Page: https://pelikan.io

License: Apache License 2.0

Dockerfile 0.08% CMake 1.44% C 51.88% Shell 0.44% Python 1.13% Rust 45.03%

cache cache-storage framework key-value memcached redis

pelikan's Introduction

Pelikan

Pelikan is a framework for developing cache services. It is:

Fast: Pelikan provides high-throughput and low-latency caching solutions.
Reliable: Pelikan is designed for large-scale deployment and the implementation is informed by our operational experiences.
Modular: Pelikan is a framework for rapidly developing new caching solutions by focusing on the inherent architectural similarity between caching services and providing reusable low-level components.

Website | Chat

Content

Overview
- Products
- Features
Build
Usage
Community
- Stay in touch
- Contributing
Documentation
License

Overview

After years of using and working on various cache services, we built a common framework that reveals the inherent architectural similarity among them.

By creating well-defined modules, most of the low-level functionalities are reused as we create different binaries. The implementation learns from our operational experiences to improve performance and reliability, and leads to software designed for large-scale deployment.

The framework approach allows us to develop new features and protocols quickly.

Products

Pelikan contains the following products:

pelikan_segcache_rs: a Memcached-like server with Segcache as the backing storage, a TTL-centric design offering extremely high memory efficiency and excellent core scalability. See our NSDI'21 paper for design and evaluation details.
pelikan_pingserver_rs: an over-engineered, production-ready ping server useful as a tutorial and for measuring baseline RPC performance
momento_proxy: a proxy which allows existing applications to use Momento instead of a Memcache-compatible cache backend.

Legacy

Pelikan legacy codebase in C can be found within the legacy folder of this project. It offers the same design blueprint as the current mainline, and implements multiple storage backend, data structures, and protocols. However, it only builds single-threaded, plain-text backends. It remains as a reference, but is not actively worked on. We do not recommend it for production deployments.

Features

runtime separation of control and data plane
predictably low latencies via lockless data structures, worker never blocks
per-module config options and metrics that can be composed easily
multiple storage and protocol implementations, easy to further extend
low-overhead command logger for hotkey and other important data analysis

Building Pelikan

Requirement

Rust stable toolchain
(to build momento_proxy) Follow the gRPC instructions here to install Protocol Buffer Compiler. Alternatively, you can skip the momento_proxy package.
C toolchain: llvm/clang (>= 7.0)
Build tools: cmake (>= 3.2)

The C/C++ tools are used by Pelikan dependencies such as BoringSSL, and also necessary if you are building legacy Pelikan which is written in C.

Build

git clone https://github.com/pelikan-io/pelikan
cd pelikan
cargo build --release
# to skip momento_proxy, run:
# cargo build --workspace --exclude momento_proxy --release

Tests

cargo test

Usage

Using pelikan_segcache_rs as an example, other executables are highly similar.

To get info of the service, including usage format and options, run:

target/release/pelikan_segcache_rs --help

To launch the service with default settings, simply run:

target/release/pelikan_segcache_rs

To launch the service with the sample config file, run:

target/release/pelikan_segcache_rs config/segcache.toml

You should be able to try out the server using an existing memcached client, or simply with telnet.

$ telnet localhost 12321
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
set foo 0 0 3
bar
STORED

Attention: use admin port for all non-data commands.

$ telnet localhost 9999
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
version
VERSION 0.1.0
stats
STAT pid 54937
STAT time 1459634909
STAT uptime 22
STAT version 100
STAT ru_stime 0.019172
...

Configuration

Pelikan is file-first when it comes to configurations, and currently is config-file only. You can create a new config file following the examples included under the config directory.

Community

Stay in touch

Join our Discord server for questions and discussions
Visit http://pelikan.io
Follow us on Twitter: @pelikan_cache

Contributing

Please take a look at our community manifesto and coding style guide.

If you want to submit a patch, please follow these steps:

create a new issue
fork on github & clone your fork
create a feature branch on your fork
push your feature branch
create a pull request linked to the issue

Documentation

We have made progress and are actively working on documentation, and will put it on our website. Meanwhile, check out the current material under docs/

License

This software is licensed under the Apache 2.0 license, see LICENSE for details.

pelikan's People

Contributors

Stargazers

Watchers

Forkers

brayniac momentohq clown-team swlynch99 beinan thinkingfish funcguy gujiacun liuchangyan punkeel sharbeargle junmingzhao42 mihirn liuxisong mbrukman

pelikan's Issues

Segcache integration test is flaky within MacOS CI

The segcache integration test regularly fails for unrelated PRs in CI. #58 has marked this test as ignored. Once this issue is resolved that should be removed.

The test failure tends to look like this

2023-02-06T15:24:31.495+00:00 INFO [admin] running admin on: 127.0.0.1:9999
2023-02-06T15:24:31.495+00:00 INFO [server::listener] running server on: 0.0.0.0:12321

2023-02-06T15:24:41.584+00:00 INFO [integration::common] testing: get miss
2023-02-06T15:24:41.586+00:00 INFO [integration::common] status: passed

2023-02-06T15:24:41.586+00:00 INFO [integration::common] testing: gets miss
2023-02-06T15:24:41.667+00:00 INFO [integration::common] status: passed

2023-02-06T15:24:41.667+00:00 INFO [integration::common] testing: set and get
2023-02-06T15:24:41.888+00:00 INFO [integration::common] status: passed

2023-02-06T15:24:41.888+00:00 INFO [integration::common] testing: cas not_found
2023-02-06T15:24:41.990+00:00 INFO [integration::common] status: passed

2023-02-06T15:24:41.990+00:00 INFO [integration::common] testing: cas exists
2023-02-06T15:24:42.433+00:00 INFO [integration::common] status: passed

2023-02-06T15:24:42.433+00:00 INFO [integration::common] testing: cas stored
2023-02-06T15:24:42.653+00:00 ERROR [integration::common] expected: [86, 65, 76, 85, 69, 32, 52, 32, 48, 32, 49, 13, 10, 48, 13, 10, 69, 78, 68, 13, 10]
2023-02-06T15:24:42.653+00:00 ERROR [integration::common] received: [69, 78, 68, 13, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
thread 'main' panicked at 'status: failed
', src/server/segcache/tests/common.rs:249:17
stack backtrace:
   0:        0x106555d08 - std::backtrace_rs::backtrace::libunwind::trace::h310cbd77a7d2ae59
                               at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
   1:        0x106555d08 - std::backtrace_rs::backtrace::trace_unsynchronized::h5768bae568840507
                               at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:        0x106555d08 - std::sys_common::backtrace::_print_fmt::hd104a205649a2ffb
                               at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/sys_common/backtrace.rs:65:5
   3:        0x106555d08 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h521420ec33f3769d
                               at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/sys_common/backtrace.rs:44:22
   4:        0x1064d829a - core::fmt::write::h694a0d7c23f57ada
                               at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/fmt/mod.rs:1208:17
   5:        0x10652f92c - std::io::Write::write_fmt::h1920a3973ad439e5
                               at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/io/mod.rs:1682:15
   6:        0x106559baa - std::sys_common::backtrace::_print::h75582c4ed1a04abb
                               at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/sys_common/backtrace.rs:47:5
   7:        0x106559baa - std::sys_common::backtrace::print::hef1aa4dbdc07ee06
                               at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/sys_common/backtrace.rs:34:9
   8:        0x1065597d3 - std::panicking::default_hook::{{closure}}::h529701a1070b4ce0
                               at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:267:22
   9:        0x10655a820 - std::panicking::default_hook::hfeeab2c667b2d7c2
                               at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:286:9
  10:        0x10655a820 - std::panicking::rust_panic_with_hook::h1b5245192f90251d
                               at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:688:13
  11:        0x10655a2e8 - std::panicking::begin_panic_handler::{{closure}}::h3658f3a9566379d4
                               at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:577:13
  12:        0x10655a278 - std::sys_common::backtrace::__rust_end_short_backtrace::h9e01645d962f8882
                               at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/sys_common/backtrace.rs:137:18
  13:        0x10655a242 - rust_begin_unwind
                               at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:575:5
  14:        0x1066725b3 - core::panicking::panic_fmt::h0097ad8ec0b07517
                               at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/panicking.rs:64:14
  15:        0x1064b91b2 - integration::common::test::h11d917b82d2bf915
  16:        0x1064b9a68 - integration::common::tests::hc09557e5b6bd7830
                               at /Users/runner/work/pelikan/pelikan/src/server/segcache/tests/common.rs:56:5
  17:        0x1064b9a68 - integration::main::hc13071f0434bfd2d
                               at /Users/runner/work/pelikan/pelikan/src/server/segcache/tests/integration.rs:28:5
  18:        0x1064b6826 - core::ops::function::FnOnce::call_once::h5eb64975d91e601d
                               at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/core/src/ops/function.rs:507:5
  19:        0x1064b6826 - std::sys_common::backtrace::__rust_begin_short_backtrace::h5a471b2b99bf64a3
                               at /rustc/fc594f15669680fa70d255faec3ca3fb507c3405/library/std/src/sys_common/backtrace.rs:121:18
  20:        0x1064bbf9c - _main

Add storage library to support datastructures

Add a new storage library which is capable of handling additional datastructures. Initially this can be implemented using a hashtable from seg and native collections (eg: std::collections::HashMap) and relying on jemalloc (or other global allocator) for memory management.

Add support for ssd/nvme storage

Add support for storing items on SSD/NVMe directly. This can be possibly be achieved by expanding the datapool crate and providing an abstraction that allows access via directio and some buffers.

Add log message when hash power is too low

Add a log message that helps users determine that the hash power is too low. It's easy to have a configuration issue there and it would be nice to suggest that they increase the hash power. We might want to log these messages only once per run, so we'd need to think about that a bit.

Other related configuration issues might be that the segment size is too small (items not fitting into segments).

Add support for tiered storage

We would like to be able to support keeping hotter items in DRAM and cooler items on SSD/NVMe.

Add basic Redis compatible backend

Tracking issue for adding a basic Redis compatible backend. Initially we should aim to support basic key-value commands by using the RESP protocol and the seg storage library.

Resp protocol parser crashes when given a command ending with a newline

To reproduce,

Start a pelikan resp service somehow (e.g. momento proxy)
Run netcat to connect to the resp port and type out
```
GET test<enter>
```

This will cause pelikan to panic and crash. The root cause is this line here: https://github.com/pelikan-io/pelikan/blob/main/src/protocol/resp/src/request/mod.rs#L89

Building Dockerfile results in missing protoc error

19.98 --- stderr
19.98 thread 'main' panicked at 'Could not find protoc installation and this build crate cannot proceed without
19.98 this knowledge. If protoc is installed and this crate had trouble finding
19.98 it, you can set the PROTOC environment variable with the specific path to your
19.98 installed protoc binary.If you're on debian, try apt-get install protobuf-compiler or download it from https://github.com/protocolbuffers/protobuf/releases

We can correct this by installing protobuf-compiler prior to running that command.

Publish on crates.io

Tracking ticket for publishing on crates.io:

src/common - pelikan-common - "Common internal libraries for Pelikan"
src/config - pelikan-config - "Component configuration for Pelikan"
src/core/admin - pelikan-core-admin - "Admin components for Pelikan"
src/core/proxy - pelikan-core-proxy - "Proxy components for Pelikan"
src/core/server - pelikan-core-server - "Server components for Pelikan"
src/core/waker - pelikan-core-waker - "Cross-thread wakers for Pelikan"
src/entrystore - pelikan-entrystore - "Maps protocol commands to Pelikan storage operations"
src/logger - pelikan-logger - "Logging framework for Pelikan"
src/net - pelikan-net - "Networking components and abstractions for Pelikan"
src/protocol/admin - pelikan-protocol-admin - "Admin protocol for Pelikan"
src/protocol/common - pelikan-protocol-common - "Shared protocol traits for Pelikan"
src/protocol/http - pelikan-protocol-http - "HTTP protocol for Pelikan"
src/protocol/memcache - pelikan-protocol-memcache - "Memcache protocol for Pelikan"
src/protocol/ping - pelikan-protocol-ping - "Ping protocol for Pelikan"
src/protocol/resp - pelikan-protocol-resp - "Redis serialization protocol for Pelikan"
src/protocol/thrift - pelikan-protocol-thrift - "Thrift binary protocol for Pelikan"
src/proxy/momento - pelikan-momento-proxy - "Pelikan proxy which allows Memcache and Redis clients to communicate with Momento" - build target: momento_proxy
src/proxy/ping - pelikan-pingproxy (?) - "Pelikan proxy which uses the ping protocol" - build target: pelikan_pingproxy
src/proxy/thrift - pelikan-thriftproxy - "Pelikan proxy which uses the Thrift binary protocol" - build target: pelikan_thriftproxy
src/queues - pelikan-queues - "Queues for cross-thread communication in Pelikan"
src/server/pingserver - pelikan-pingserver - "A simple ASCII ping/pong server" - build target: pelikan_segcache
src/server/segcache - pelikan-segcache - "A Memcache compatible server which uses segment-structured storage" - build target: pelikan_segcache
src/session - pelikan-session - "An abstraction on-top of streams for buffered bi-directional communication"
src/storage/bloom - pelikan-storage-bloom - "Pelikan bloom filter storage"
src/storage/datapool - pelikan-storage-datapool - "An abstraction which provides byte-addressable storage"
src/storage/seg - pelikan-storage-seg - "Pelikan segment-structured storage"
src/storage/types - pelikan-storage-types - "Typed storage for use in Pelikan"

Open questions/observations:

do we want to publish the thrift proxy? (thinking no)
should src/queues move to rustcommon? (thinking yes)
should we use pelikan_ping_proxy or pelikan_pingproxy for "generic" proxy naming patterns?
I believe we do want to use pelikan_segcache for the binary name (note: dropping "_rs" suffix) and allow install via cargo install pelikan-segcache
should src/net or src/session move out to rustcommon? They feel like they could be useful in projects like rpc-perf, though we can depend on them through the pelikan- crates.
Momento proxy may

And some hazards to consider:

crates cannot be deleted, if we want to rename we will pollute our "namespace"
namespaces do not exist, anybody could write and publish a pelikan- crate
while the crate name (used for cargo install) does not need to match the name of the binary(s) that it provides, we should strive to make it easy for users

Add support for unix domain sockets

Add support to the server core for unix domain sockets. This could allow applications to use Pelikan as a host-local cache without the overhead of the tcp stack.

`Set` command option's like 'EX 1000' and NX failing to parse

When sending a Redis command like
SET foo bar NX EX 1000
or
SET foo2 bar2 EX 1000 NX

against a container running Redis I am able to set this key ok. But when I run it against the proxy code I am seeing the proxy failing to parse the more complex set command and logs the following error:
bad request: bulk string is not a u64

Add fuzzer for RESP protocol

We should really have a fuzzer for the RESP protocol implementation.

Question about segcache eviction policy

Hi @brayniac @thinkingfish @kevyang, recently I am testing segcache, and I would like to ask about the eviction policy for segcache, are both ttl and segment size factors that will trigger eviction for cache item?
For example, I set 500s ttl, and my test would take about 1 minute to run. If I am still getting 0s for a cached key, is that caused by the cache segment is full and that value is kicked out? Thank you.

Handle `SIGINT` in segcache

Currently, a containerised segcache process does not handle SIGNINT. This means to stop the container one has to force-kill it.

It would be great if the code is changed to gracefully handle SIGINT and stops the segcache server. This needs SIGNIT signal handling at Rust level for Server.Process.

benchmark: part of threads never stopped when run with memtier_benchmark

We are measuring pelikan performance with memtier_benchmark, but encounter the following issue.

Problem description

Run performance test with memtier_benchmark, when --test-time is reached, the output from memtier_benchmark indicates some threads never stop, and it has no chance to print the statistical data finally.

How to re-produce

Configuration:
The problem happens with the following settings. If memtier_benchmark threads < 20, there is no such problem.

- memtier_benchmark: run with threads >= 20.
- Pelikan: [worker] threads in configure/segcache > 5.

Commands to start test:

> Start pelikan: target/release/pelikan_segcache_rs config/segcache.toml
> Start memtier_benchmark: memtier_benchmark -p 12321 -P memcache_text -d 256 --threads 20 --test-time 10

With some debugging and code analysis, we see that those threads are looping in epoll_pwait(), waiting for events from pelikan. The soft stack: client_group::run() (memtier_benchmark/client.c) -> event_base_dispatch() (libevent/event.c) -> event_base_loop() -> epoll_dispatch() (libevent/epoll.c) -> epoll_pwait2() (kernel systemcall)

Question

Why pelikan does not send back the events which cause the blocking?
What performance tools Pelikan use for performance evaluation and without the above issue?

Add support for drop-in replacement of guava

It would be nice to offer the ability to use the seg storage library as a drop-in replacement for Guava. We should create a compatible library so that only relevant imports need to be changes to use Pelikan instead