Giter VIP home page Giter VIP logo

uhlc-rs's Introduction

uhlc-rs

build crate API

A Unique Hybrid Logical Clock for Rust.

This library is an implementation of an Hybrid Logical Clock (HLC) associated to a unique identifier. Thus, it is able to generate timestamps that are unique across a distributed system, without the need of a centralized time source.

Usage

Add this to your Cargo.toml:

[dependencies]
uhlc = "0.8"

Then in your code:

use uhlc::*;

// create an HLC with a random u128 and relying on SystemTime::now()
let hlc = HLC::default();

// generate a timestamp
let ts = hlc.new_timestamp();

// update the HLC with a timestamp incoming from another HLC
if ! hlc.update_with_timestamp(&other_ts).is_ok() {
    println!(r#"The incoming timestamp would make this HLC
             to drift too much. You should refuse it!"#);
}

What is an HLC ?

A Hybrid Logical Clock combines a Physical Clock with a Logical Clock. It generates monotonic timestamps that are close to the physical time, but with a counter part in the last bits that allow to preserve the "happen before" relationship.

You can find more detailled explanations in:

Why "Unique" ?

In this implementation, each HLC instance is associated with an identifier that must be unique accross the system (by default a random u128). Each generated timestamp, in addition of the hybrid time, contains the identifier of the HLC that generated it, and it therefore unique across the system.

Such property allows the ordering all timestamped events in a distributed system, without the need of a centralized time source or decision.

Note that this ordering preserve the "happen before" relationship only when events can be correlated. I.e.:

  • if 2 events have the same source, no problem since they will be timestamped by the same HLC that will generate 2 ordered timestamps.

  • if an entity receives an event with a timestamp t1, it must update its HLC with t1. Thus, all consecutive generated timestamps will be greater than t1.

  • if 2 events have different sources that have not exchanged timestamped events before, as the physical clocks on each source might not be synchronized, it may happen that the HLCs generate timestamps that don't reflect the real physical ordering. But in most cases this doesn't really matter since there is no a real correlation between those events (one is not a consequence of the other).

Implementation details

The uhlc::HLC::default() operation generate a random u128 as identifier and uses std::time::SystemTime::now() as physical clock.
But using the uhlc::HLCBuilder allows you to configure the HLC differently. Example:

let custom_hlc = HLCBuilder::new()
    .with_id(ID::try_from([0x01, 0x02, 0x03]).unwrap())     // use a custom identifier
    .with_clock(my_custom_gps_clock)                        // use a custom physical clock (e.g. using GPS as time source)
    .with_max_delta(Duration::from_secs(1))                 // use a custom maximum delta (see explanations below)
    .build();

A uhlc::HLC::NTP64 time is 64-bits unsigned integer as specified in RFC-5909. The first 32-bits part is the number of second since the EPOCH of the physical clock, and the second 32-bits part is the fraction of second. In case its generated by an HLC, the last few bits of the second part are replaced by the HLC logical counter. The size of this counter currently hard-coded to 4 bits in uhlc::CSIZE.
This gives a theoretical time resolution of (0xF * 10^9 / 2^32) = 3.5 nanoseconds.

To avoid a "too fast clock" to make an HLC drift too much in the future, the uhlc::HLC::update_with_timestamp(timestamp) operation will return an error if the incoming timestamp exceeds the current physical time more than a delta (500ms by default, configurable declaring the UHLC_MAX_DELTA_MS environment variable). In such case, it could be wise to refuse or drop the incoming event, since it might not be correctly ordered with further events.

Cargo features

This crate provides the following Cargo features:

  • std: allows this crate to use the full std. Even if disabled, notice that the alloc crate is still required;

  • defmt: allows the relevant data structures to implement the defmt::Format trait, used instead of std::fmt::{Debug, Display} for logging in no_std environments.

Only the std feature is enabled by default.

Usage in no_std environments

In order to use this crate in a no_std environment, the default-features = false flag should be added in the dependencies section of the Cargo.toml file. The main differences with respect to the std implementation include:

  • environment variables do not exist in an embedded environment, hence UHLC_MAX_DELTA_MS cannot be used to tweak at runtime the delta for the clock "anti-drift" mechanism. An appropriate value must always be set at compile time;

  • usually, embedded systems do not keep track of "real world" time, but re-initialize their hardware timers every time they boot. Hence, the physical clock that is used when calling uhlc::HLC::default() is a dummy function that always return a zero-valued timestamp. Since the HLC is responsible for ensuring that timestamps are strictly increasing in order to preserve the "happen before" relationship, this means calls to uhlc::HLC::new_timestamp() return incremental integers;

  • for the same reason, parsing from and formatting to human-readable time formats is not available in no_std;

  • the std::sync::Mutex (internally used to guarantee timestamps monotonicity) is replaced by spin::Mutex, which is based on spinlocks instead of relying on some operating system functionality;

  • tests (with cargo test) can be run only on std targets, but different code is compiled (and hence tested) depending on the features specified.

Usages

uhlc is currently used in Eclipse zenoh.

uhlc-rs's People

Contributors

davidedellagiustina avatar dylan-dpc avatar gabrik avatar j-loudet avatar jenoch avatar mallets avatar olivierhecart avatar p-avital avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

uhlc-rs's Issues

Timestamp is not round-trip equivalent with Display and parse()

The + 1 at the end of the impl From<Duration> breaks stringifying and then parsing the same number back out.

NTP64((secs << 32) + ((nanos * FRAC_PER_SEC) / NANO_PER_SEC) + 1)

The following test fails:

    #[test]
    fn test_exact_parse() {
        use std::str::FromStr;
        let id1: ID = ID::try_from([0x01]).unwrap();
        let ts1 = Timestamp::new(NTP64(100), id1);
        assert_eq!(ts1, Timestamp::from_str(&ts1.to_string()).unwrap());
    }

Request: Allow zero in `ID`.

In my project, which uses HLCs, a "client" will assign themselves a placeholder ID of 0 until they have been assigned an ID when syncing data with a remote server. Now that this crate's ID representation has changed from a NonZeroU128 to a [u8; 16], is there a reason that an ID with value 0 should be disallowed?
I skimmed through the code and couldn't find anything that would disallow 0, other that the From/TryFrom implementations for ID.

Parsing an ID is sometimes broken with a valid ID

I have a valid uuid::UUID that can be converted to a valid uhlc::ID (I used try_into()), but I can't parse the timestamp once it has been serialized.

Test case:

#[cfg(test)]
mod tests {
    #[test]
    fn uhlc_ids() {
        let uuid: uuid::Uuid = "0478B15B80D54EE884B969D276E5B000".parse().unwrap();
        let clock = uhlc::HLCBuilder::new()
            .with_id(uuid.as_u128().try_into().unwrap())
            .build();

        let ts = clock.new_timestamp();

        println!("ts: {ts}");

        let ts: uhlc::Timestamp = ts.to_string().parse().unwrap();

        println!("parsed ts: {ts}");
    }
}

Produces:

ts: 2023-08-24T19:16:28.660307999Z/0b0e576d269b984e84ed5805bb17804
thread 'broadcast::tests::uhlc_ids' panicked at 'called `Result::unwrap()` on an `Err` value: ParseTimestampError { cause: "Leading 0s are not valid" }', crates/corro-types/src/broadcast.rs:367:58
stack backtrace:
   0: rust_begin_unwind
             at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/panicking.rs:578:5
   1: core::panicking::panic_fmt
             at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/core/src/panicking.rs:67:14
   2: core::result::unwrap_failed
             at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/core/src/result.rs:1687:5
   3: core::result::Result<T,E>::unwrap
             at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/core/src/result.rs:1089:23
   4: corro_types::broadcast::tests::uhlc_ids
             at ./src/broadcast.rs:367:35
   5: corro_types::broadcast::tests::uhlc_ids::{{closure}}
             at ./src/broadcast.rs:357:19
   6: core::ops::function::FnOnce::call_once
             at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/core/src/ops/function.rs:250:5
   7: core::ops::function::FnOnce::call_once
             at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
test broadcast::tests::uhlc_ids ... FAILED

I believe this is because some valid IDs can start with zero?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.