Technical Debt

• Write unit tests for event messaging pattern
• remove src/bin/* and old rust_experiments
• namespace/prefix for config files e.g. /tmp/elkodon
• replace all iceoryx´ strings with elkodon`
• follow XDG spec
  • https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html
  • https://www.freedesktop.org/wiki/Software/xdg-user-dirs/
  • set path in pal/settings
• remove platform suffix in pal/settings
• check where &[u8] can be replaced with &str
• tests for discovery
• remove elkodon_bb_posix::SignalHandler from public API and examples, introduce eventloop::wait() instead (maybe with better name)
• final API review before the release in case there are still some quirks left
  • no get_ prefix for getter but a set_ prefix for setter

Cargo.toml

• add version inheritance from root workspace
• Add the following entries:
• every package inherits from main workspace and does not add new dependencies

[workspace.package]
version = ""
repository = ""
homepage = ""
authors = []
edition = "2021"
license = ""
categories = ["ipx"]
description = "elkodon: zero copy ipc"

Changelog

• introduce CHANGELOG.md
• evaluate CI check that every pull request comes with a changelog update

Contributions and Conventions

Write contribution md with the content:

• name conventions for release branch, tags
• branch protection for release branches
• Introduce CONTRIBUTORS.md
• Add pull request template
• Add issue and bug issue template
• elkodon vs Elkodon

Licensing & File Header

• Introduce script to check and modify license header
• Use only spdx license code as header

Documentation

• README.md that contains supported platforms, example, overview
• ROADMAP.md
  • add waitset and quality of life improvements
• CONTRIBUTIONS.md
• documentation for every example
• document whole elkodon crate
• Add badges for CI, Code Coverage, Documentation state, gitter, discord, license, https://linktr.ee/, create Reddit iceoryx room
• Add github labels
• Document feature flags in the README.md, see tracing and log
• Document feature flags in elkodon/lib.rs, see tracing and log

CI Checks

• commit message style elk-##ISSUE_NUMBER##-##BRANCH_NAME##
• every commit is signed
• that linked issue exists
• one approval required
• Documentation contains no warnings
• add ci check that new crate dependencies are only added in the root Cargo.toml and the subcrates inherit from it

CI Platform checks

• Linux x86_64 (archlinux)
• Linux x86_64 (ubuntu-latest)
• Linux aarch64 (archlinux)
• Linux aarch64 (ubuntu-latest)
• Linux x86_64 (rust-nightly)
• FreeBSD
• Windows
• Mac OS

Permanent Issues

• CI updates
• Documentation fixes

Brief feature description

Currently, when creating/opening a service with an existing name but a different messaging pattern one receives the failure that it either exists or that the messaging pattern does not match.

It would be better if this would be allowed and that a service is uniquely identified by the messaging pattern and the name.

Detailed information

Should be easily implementable, just hash the messaging pattern as well to acquire the services uuid.

Brief feature description

Currently, the elkodon logger uses its own log backend as default. To allow it to integrate seamlessly into other frameworks, the major log frameworks log and tracing shall be supported via feature flags.

ToDo

• Implement log and tracing sinks
• Document feature flags in the README.md
• Document feature flags in elkodon/lib.rs

In the elkodon.toml file an additional directory (/tmp/elkodon/ as root path instead /tmp) shall be created to prevent clashes with other applications. More high-level constructs are then required to create subdirectories like /tmp/elkodon/services when neither the dir elkodon or services exist (recursive subdirectory creation).

When elkodon_pp_posix::Directory::create() would create them recursively the issue would be fixed otherwise these higher constructs get the error that part of the path do not exist.

Brief feature description

From #25 (comment)

========

@elfenpiff I found this https://stackoverflow.com/questions/35988775/how-can-i-cause-a-panic-on-a-thread-to-immediately-end-the-main-thread/36031130#36031130

The following code would terminate the whole test when an assert in a thread fails. It can also be placed in a function which could be called at the start of a test

let orig_hook = panic::take_hook();
panic::set_hook(Box::new(move |panic_info| {
    // invoke the default handler and exit the process
    orig_hook(panic_info);
    process::exit(1);
}));

========

(Code) Example Of Cumbersome API

Currently, it can happen that due to some API adjustments code examples in the markdown files are no longer running. The Geoffrey solves this problem.

Improvement Suggestion

Only provide code examples in combination with geoffrey in markdown files.

When executing the discovery example, all services that are listed are removed.

In ubuntu 22.04 the call getsockopt with SO_RCVBUF does not return the actual receive buffer size. The tests have to be adjusted that the internal buffer size of ubuntu is capped to 11.

Hi,
the code looks promising. However, it is now licensed as GPL.
Would it be possible to change the license to MPL?

I think MPL would allow wide adoption while still ensuring contributions come back to the main source.
Thank you for considering.

cargo doc creates warnings despite a successful cargo test --workspace --doc run.

• Fix doc failures
• Add cargo doc to CI run

Required information

Operating system:

• OS Windows

Rust version:
Output of: rustc 1.73

Cargo version:
Output of: cargo 1.73

elkodon version:
main branch

Observed result or behaviour:
The test publisher_block_when_unable_to_deliver_blocks in crate elkodon deadlocks sometimes in windows

Brief feature description

The ability to quickly communicate with the maintainers of elkodon or to promote new features requires some kind of social media accounts. Explore what kind of mechanisms can be used and create an account.

• gitter?
• youtube channel?
• discord
• mastodon

Required information

Operating system:

• OS Windows 2019 Server

Rust version:
Output of: rustc 1.73

Cargo version:
Output of: cargo 1.73

elkodon version:
main branch

Observed result or behaviour:
In elkodon_pal/posix/src/windows/mman.rs the shm_unlink call uses DeleteFileA. When a file is opened and deleted with this call it successfully returns. If DeleteFileA is again called while the file is opened it will set the GetLastError to ERROR_ACCESS_DENIED instead of ERROR_FILE_NOT_FOUND, see: https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-deletefilea

This causes all elkodon_cal tests to fail that are using shared memory. Whenever std::mem::forget is called in such a concept, the file descriptor is leaked and a subsequent second remove call will return permission denied instead of false.

The quick-fix Errno::set(Errno::ENOENT); in elkodon_pal/posix/src/windows/mman.rs (shm_unlink) shall be removed and the concept tests must be adjusted.

(Code) Example Of Cumbersome API

Currently, the default config file for windows is named config_win.toml to handle the windows path style. The unix variant is named config.toml.

Both should be named config.toml but the problem is, how to deliver two different default config files one for windows and one for the rest of the systems.

Brief feature description

The error handling strategy must be documented.

• That we use enums as error codes
• A strategy how internal error codes from a lower architectural level are translated into enum error codes of an higher architectural level
• How std::error::Error and later (when it is in stable) core::error::Error is implemented for all error enums.

The issue shall be concluded with a markdown file explaining the above points and showing some code snippets to illustrate them.

Detailed information

From @elBoberido

I found another solution in the zstd crate. They use derive_more with a feature flag to derive Error only on std. This makes would allow no_std on stable but without the Error trait. Something to think of.

They basically did this

[dependencies]
derive_more = { version = "0.99", default-features = false, features = ["display", "from"] }

[features]
default = ["std"]
std = ["derive_more/error"]

#[derive(Debug, derive_more::Display, derive_more::From)]
#[cfg_attr(feature = "std", derive(derive_more::Error))]
#[non_exhaustive]
pub enum Foo {
    #[display(fmt = "Bar occurred. Is: {baz}, must be either 1 or 2")]
    Bar { baz: u8 },
    //...
}

The attributes on the enum tags are quite similar to thiserror so it wouldn't be too hard to switch between the two crates.

To be able to check certain CI failures it is important to recreate the exact environment. This is the exact puprose of ice_env.sh in iceoryx.
The script can be adjusted so that it works also for elkodon.

Required information

Observed result or behaviour:
It is possible to loan a sample and immediately publish it without initializing the data.

Expected result or behaviour:
With safe Rust this should not compile.

Conditions where it occurred / Performed steps:
Compile this code

let sample = publisher.loan()?;
publisher.send(sample)?;

Proposal

I think we can borrow some code from iceoryx-rs and let the API for uninitialized samples return a SampleMut<MaybeUninit<T>> instead of a SampleMut<T>. The user would have to call the unsafe assume_init method in order to get a SampleMut<T> to be able to publish the sample.

This is the iceoryx-rs API for uninitialized samples

let mut sample = publisher.loan_uninit()?;
let sample = unsafe {
    (*sample.as_mut_ptr()).counter = counter;
    sample.assume_init()
};
publisher.publish(sample);

This would lead to a compile time error

let sample = publisher.loan_uninit()?;
publisher.publish(sample);

Brief feature description

When an application crashes, it can happen that a service is only partially removed and becomes unusable.
An internal service and/or a command line application shall cleanup the remainders of such services.

The following use cases shall be considered:

• remove static service info, when only the dynamic service info exists
• remove dynamic service info, when only the static service info exists
• remove the data segment of a publisher if the owning process no longer exists
• remove the publisher/subscriber as participant from the dynamic service info when the owning process no longer exists.

Brief

This issue takes care of small documentation fixes like typos or out-of-date examples/explanations.

Required information

If a thread in a test encounters an assert and suddenly terminates, it is possible that some succeeding variables, required for terminating the whole tests, are not being set. Therefore, either avoid asserts in tests or use it only if it occurs not before such a test flow control variable.

Required information

Operating system:

• OS Ubuntu 22.04 LTS aarch64

Rust version:
Output of: rustc 1.73.0

Cargo version:
Output of: cargo 1.73.0

elkodon version:
main branch

Observed result or behaviour:
The test access_control_list_string_conversion_works fails.

Expected result or behaviour:
The test access_control_list_string_conversion_works passes.

Brief feature description

Currently, the publish-subscribe benchmark is working in one process between two threads. The benchmark shall be extended so that it can also run between two distinct processes.

Brief feature description

Fix shared memory list issue and activate windows on the cirrus ci.

(Code) Example Of Cumbersome API

If one would like to print a state overview of a service it would be nice if the port_factory::{event|publish_subscribe|...} would have implemented Display to show the overview from the example.

Brief feature description

Explore if the bot from the article: https://www.cncf.io/blog/2023/06/06/a-chatgpt-powered-code-reviewer-bot-for-open-source-projects/ can be applied to elkodon

Brief feature description

A pub-sub service has the ability to define a max buffer size for a subscriber but the subscriber is unable to reduce its buffer size - every subscriber has the same buffer size.

So we need to introduce a setting in the subscriber port factory to set a custom buffer size.

Brief feature description

If the example code is also run, we can ensure that the example is also functional working. Therefore, we should remove the no_run tag from as many documentation code examples as possible.

Brief

The CI unveiled a few issues with some tests. Those tests are temporarily skipped and need to be re-enabled.

Just seach for 'TODO elk-#30:' to find all the sipped tests.

Problem

When two elkodon_cal constructs interact with each other, we need to add some initial versioning information to ensure that not to incompatible versions of the same constructs are interacting. This could lead to undefined behavior and hard to unravel bugs.

They could be, for instance incompatible since the member order of shared management constructs has changed or members have been added/removed. Or the internal communication protocol was updated.

Solution

Introduce a const SHM_API_VERSION: u64 constant inside elkodon_cal, that must be checked before any of these constructs start to interact on an ipc level.
Constructs that use shared memory could add this constant to the beginning, as the first value, to their shared memory. If the construct does not use shared memory, one could create a file and store the number value is binary, or human-readable value, in there.

Brief

This issue takes care of setting up the CI and maintaining it

Brief feature description

Currently, the enable_safe_overflow behavior is part of the service but it should be actually a subscriber property for its buffer behavior.