All the backends/storages implementation (except the built-in memory backend) will have it's own repo.
This will allow:

• independant release cycle of the backend wrt. zenoh's
• no impact on zenoh build of 3d-party dependencies that the backend may have (especially wrt. portability)
• shorter build time for zenoh itself

Add a backend for InfluxDB that will:

• allow creation of storages, each mapping to a database
• store each zenoh path/value publication as a point where:
  • each path will correspond to a measurement
  • each value will be converted as a string to be stored as point's value
  • the the zenoh's timestamp's time will be used as InfluxDB timestamp
  • the zenoh's encoding and the full timestamp will be set as tags within the point
• support selectors with starttime and stoptime properties, using those as time in WHERE clauses

Create a suite of performance tests for zenoh

Description

Branch: https://github.com/eclipse-zenoh/zenoh/tree/rust-master
Zenoh Rust Version: 21b346a

I was using the zenoh-ffi to implement a couple of pubsub nodes (for use with ROS2), and discovered some issues with Zenoh.

These issues break any implementations that use the FFI where publish and subscribes happen in the same process, but on the same network as other publishers or subscribers in different processes.

Namely, for processes with both Zenoh publishes and subscribes:

1. There are non-deterministic instances where messages that are sent by the process are not received in the same process if another process is actively subscribed to another Zenoh topic (related or not), but the message can get received by another process.
2. Callbacks for messages sent by a process are non-deterministically mismatched when received by the same process, if there is another process actively subscribed to another Zenoh topic (related or not.)

Note that these issues are non-deterministic. Run the examples a couple of times, and they'll pop up.

Additional note: I am writing to the topics using zn_write_wrid()

Examples

I have prepared a minimal package to exhibit this behaviour. Please find it here and ensure you are on the cmake-only branch.

Build instructions are found in the README.

NOTE: The processes in each example are started in order.
And the resource IDs as well as the topic keys are printed for convenience.

Example 1: Dropped Intra-process Messages

If a single pubsub process is started, everything works as expected.

zenoh_minimal_cpp/build/bin$ ./test_pubsub /test
Subscription expression to /test (1)

<<< Published 36 bytes to /test: 'Message #0 from /test (1) to topic /test'
>>> Received 36 bytes on /test: 'Message #0 from /test (1) to topic /test'

<<< Published 36 bytes to /test: 'Message #1 from /test (1) to topic /test'
>>> Received 36 bytes on /test: 'Message #1 from /test (1) to topic /test'

<<< Published 36 bytes to /test: 'Message #2 from /test (1) to topic /test'
>>> Received 36 bytes on /test: 'Message #2 from /test (1) to topic /test'

However, once we start a subscriber and attach it to an unrelated topic before the pubsub process, it sometimes fails to receive the message.

Process 1: Unrelated Subscriber

zenoh_minimal_cpp/build/bin$ ./test_sub /topic
Subscription expression to /topic

Process 2: Pubsub

zenoh_minimal_cpp/build/bin$ ./test_pubsub /test
Subscription expression to /test (1)

<<< Published 36 bytes to /test: 'Message #0 from /test (1) to topic /test'

<<< Published 36 bytes to /test: 'Message #1 from /test (1) to topic /test'

<<< Published 36 bytes to /test: 'Message #2 from /test (1) to topic /test'

More perplexingly, if another subscriber (this time subscribed to the appropriate topic that is being published to) is started after the first unrelated subscriber, it sometimes is able to receive the message even though the pubsub process is unable to (this means that it probably isn't an issue with the size_t topic IDs that are assigned to each key expression on resource declaration.)

(Also note that this only happens sometimes, indicating a possible race condition.)

Process 1: Unrelated Subscriber

zenoh_minimal_cpp/build/bin$ ./test_sub /topic
Subscription expression to /topic

Process 1: Related Subscriber

zenoh_minimal_cpp/build/bin$ ./test_sub /test
Subscription expression to /test
>>> Received 36 bytes on /test: 'Message #0 from /test (1) to topic /test'
>>> Received 36 bytes on /test: 'Message #1 from /test (1) to topic /test'
>>> Received 36 bytes on /test: 'Message #2 from /test (1) to topic /test'

Process 3: Pubsub

zenoh_minimal_cpp/build/bin$ ./test_pubsub /test
Subscription expression to /test (1)

<<< Published 36 bytes to /test: 'Message #0 from /test (1) to topic /test'

<<< Published 36 bytes to /test: 'Message #1 from /test (1) to topic /test'

<<< Published 36 bytes to /test: 'Message #2 from /test (1) to topic /test'

Example 2: Mixed Callbacks

If we initialise an unrelated subscriber, a related subscriber, and a pubsub process that publishes to multiple topics, only the pubsub receives subscriber callbacks with mismatched topics, but the related subscriber receives them correctly.

NOTE: This issue seems to disappear if we do not spin up the unrelated subscriber.

The order for starting the subscribers does not seem to matter. All that matters is the presence of the unrelated subscriber.

(You can try starting two pubsub processes on the same topic as well and you should sometimes see half of the messages get their callbacks mixed up (from the messages coming from the same process.))

Process 1: Unrelated Subscriber

zenoh_minimal_cpp/build/bin$ ./test_sub /unrelated
Subscription expression to /unrelated

Process 2: Related Subscriber

zenoh_minimal_cpp/build/bin$ ./test_sub /topic_a /topic_b /topic_c
Subscription expression to /topic_a
Subscription expression to /topic_b
Subscription expression to /topic_c
>>> Received 46 bytes on /topic_c: 'Message #0 from /topic_c (3) to topic /topic_c'
>>> Received 46 bytes on /topic_b: 'Message #0 from /topic_b (2) to topic /topic_b'
>>> Received 46 bytes on /topic_a: 'Message #0 from /topic_a (1) to topic /topic_a'
>>> Received 46 bytes on /topic_c: 'Message #1 from /topic_c (3) to topic /topic_c'
>>> Received 46 bytes on /topic_b: 'Message #1 from /topic_b (2) to topic /topic_b'
>>> Received 46 bytes on /topic_a: 'Message #1 from /topic_a (1) to topic /topic_a'
>>> Received 46 bytes on /topic_a: 'Message #2 from /topic_a (1) to topic /topic_a'
>>> Received 46 bytes on /topic_b: 'Message #2 from /topic_b (2) to topic /topic_b'
>>> Received 46 bytes on /topic_c: 'Message #2 from /topic_c (3) to topic /topic_c'

Process 3: Multi-Pubsub

zenoh_minimal_cpp/build/bin$ ./test_pubsub /topic_a /topic_b /topic_c
Subscription expression to /topic_a (1)
Subscription expression to /topic_b (2)
Subscription expression to /topic_c (3)

<<< Published 46 bytes to /topic_c: 'Message #0 from /topic_c (3) to topic /topic_c'
>>> Received 46 bytes on /topic_b: 'Message #0 from /topic_c (3) to topic /topic_c'	<---- Mismatched topics
<<< Published 46 bytes to /topic_b: 'Message #0 from /topic_b (2) to topic /topic_b'
>>> Received 46 bytes on /topic_a: 'Message #0 from /topic_b (2) to topic /topic_b'	<---- Mismatched topics
<<< Published 46 bytes to /topic_a: 'Message #0 from /topic_a (1) to topic /topic_a'

<<< Published 46 bytes to /topic_c: 'Message #1 from /topic_c (3) to topic /topic_c'
<<< Published 46 bytes to /topic_b: 'Message #1 from /topic_b (2) to topic /topic_b'
<<< Published 46 bytes to /topic_a: 'Message #1 from /topic_a (1) to topic /topic_a'
>>> Received 46 bytes on /topic_a: 'Message #1 from /topic_b (2) to topic /topic_b'	<---- Mismatched topics
>>> Received 46 bytes on /topic_b: 'Message #1 from /topic_c (3) to topic /topic_c'	<---- Mismatched topics

<<< Published 46 bytes to /topic_c: 'Message #2 from /topic_c (3) to topic /topic_c'
<<< Published 46 bytes to /topic_b: 'Message #2 from /topic_b (2) to topic /topic_b'
>>> Received 46 bytes on /topic_b: 'Message #2 from /topic_c (3) to topic /topic_c'	<---- Mismatched topics
>>> Received 46 bytes on /topic_a: 'Message #2 from /topic_b (2) to topic /topic_b'	<---- Mismatched topics
<<< Published 46 bytes to /topic_a: 'Message #2 from /topic_a (1) to topic /topic_a'

Misc

I am tagging @codebot and @gbiggs here so they can follow the issue as well.

- CH3EERS!

It'd be great if the Rust rewrite of zenoh compiled with Rust stable, the following are packages in zenoh that use features from nightly or that are declared unstable:

• plugins/example-plugin
  • async-std unstable features
• plugins/zenoh-http
  • async-std unstable features
  • async_closure (stabilization being tracked at rust-lang/rust#62290)
• zenoh
  • async-std unstable features
  • async_closure (stabilization being tracked at rust-lang/rust#62290)
• zenoh-ffi
  • async-std unstable features
• zenoh-perf
  • async-std unstable features
• zenoh-protocol
  • async-std unstable features
• zenoh-router
  • async-std unstable features
  • get_mut_unchecked (stabilization being tracked at rust-lang/rust#63292)
  • async_closure (stabilization being tracked at rust-lang/rust#62290)
• zenoh-util
  • async-std unstable features
  • is_sorted (stabilization being tracked at rust-lang/rust#53485)

The zenoh protocol gives router and peer the freedom to do resource generalisation. This means that a peer that has a subscriber for /foo/bar/a, a subscriber for /foo/bar/b** and a subscriber for a subscriber for /foo/bar/c is free to generalise this to advertising a generalised subscriber for a subscriber for /foo/bar/*.

In some instances it is handy, based on knowledge of an application to force resource generalisation. Thus it would be desirable to provide a way to define the resources prefix that should be generalised and to what they should be generalised.

For instance, we could say that anything starting with /foo/bar should be generalised to /foo/bar/* and something starting with /car/1A23SZ should be generalised to /car/1A23SZ/**.

I think we should maintain generalisation that identify convex sets, in other terms w/o holes.

Rational:
The HTTP plugin currently implements the REST API. The body of the responses to GET operations always contain a JSON list of a (key, value, time) struct.
We also would like to offer the possibility for an HTTP GET request on zenoh to receive only the value for a key, not wrapped in a JSON type, but directly with the value's format (depending its encoding). This feature will be implemented in a new "Web plugin".
To avoid confusion between this new "Web plugin" and the "HTTP plugin" we rename this latest as "REST plugin".

As discussed in gitter I explain my problem here:

I want to linux machines to exchange ROS2 (DDS) Topics.
One of them is behind a NAT.

On the robot is some ROS software already running when starting zenoh.
But when I run ros2 topic echo /some_data on my notebook, I don't receive any data...

How I tried:

Server:

IP=10.212.20.70

docker run -d --init --name zenoh-router --net=host eclipse/zenoh
docker run --init --name zenoh-bridge --net=host -e RUST_LOG=debug eclipse/zenoh-bridge-dds

  docker run --init --name zenoh-bridge --net=host -e RUST_LOG=debug eclipse/zenoh-bridge-dds
 * Starting: dzd 
1617292481.565764 [0]        dzd: using network interface ens192 (udp/10.212.20.70) selected arbitrarily from: ens192, docker0, br-18995b57279d, br-470c5c555947, br-5aabce492b94
[2021-04-01T15:54:41Z DEBUG zenoh::net] Zenoh Rust API v3487f6a
[2021-04-01T15:54:41Z DEBUG zenoh::net] Config: local_routing=false;mode=client
[2021-04-01T15:54:41Z INFO  zenoh::net::runtime] Using PID: 3F9EABFDE12B4DC09E5EA7B24C957C97
[2021-04-01T15:54:41Z INFO  zenoh::net::runtime::orchestrator] Scouting for router ...
[2021-04-01T15:54:41Z DEBUG zenoh::net::runtime::orchestrator] UDP port bound to 10.212.20.70:35271
[2021-04-01T15:54:41Z INFO  zenoh::net::runtime::orchestrator] Found Hello { pid: Some(94055E6C69404CC3B678B25E1B4C9B82), whatami: None, locators: Some([Tcp(SocketAddr(10.212.20.70:7447)), Tcp(SocketAddr(172.17.0.1:7447)), Tcp(SocketAddr(192.168.32.1:7447)), Tcp(SocketAddr(172.19.0.1:7447)), Tcp(SocketAddr(172.18.0.1:7447))]) }
[2021-04-01T15:54:41Z DEBUG zenoh::net::protocol::session::authenticator::shm] Peer 94055E6C69404CC3B678B25E1B4C9B82 did not express interest in shared memory
[2021-04-01T15:54:41Z DEBUG zenoh::net::protocol::session::manager] New session opened with 94055E6C69404CC3B678B25E1B4C9B82: whatami 1, sn resolution 268435456, initial sn tx 48186762, initial sn rx 117788447, is_local: false
[2021-04-01T15:54:41Z DEBUG zenoh::net::routing::router] New Face{0, 94055E6C69404CC3B678B25E1B4C9B82}
[2021-04-01T15:54:41Z DEBUG zenoh::net::runtime::orchestrator] Successfully connected to newly scouted Hello { pid: Some(94055E6C69404CC3B678B25E1B4C9B82), whatami: None, locators: Some([Tcp(SocketAddr(10.212.20.70:7447)), Tcp(SocketAddr(172.17.0.1:7447)), Tcp(SocketAddr(192.168.32.1:7447)), Tcp(SocketAddr(172.19.0.1:7447)), Tcp(SocketAddr(172.18.0.1:7447))]) }
[2021-04-01T15:54:41Z DEBUG zenoh::net::routing::router] New Face{1, 3F9EABFDE12B4DC09E5EA7B24C957C97}
[2021-04-01T15:54:41Z DEBUG zenoh::net::routing::resource] Register resource /@/router/94055E6C69404CC3B678B25E1B4C9B82/plugin/storages/backend
[2021-04-01T15:54:41Z DEBUG zenoh::net::routing::resource] Register resource /@/router/94055E6C69404CC3B678B25E1B4C9B82/plugin/storages/backend/*
[2021-04-01T15:54:41Z DEBUG zenoh::net::routing::pubsub] Register subscription /@/router/94055E6C69404CC3B678B25E1B4C9B82/plugin/storages/backend/* for Face{0, 94055E6C69404CC3B678B25E1B4C9B82}
[2021-04-01T15:54:41Z DEBUG zenoh::net::routing::resource] Register resource /@/router/94055E6C69404CC3B678B25E1B4C9B82/plugin/storages/backend/memory/storage
[2021-04-01T15:54:41Z DEBUG zenoh::net::routing::resource] Register resource /@/router/94055E6C69404CC3B678B25E1B4C9B82/plugin/storages/backend/memory/storage/*
[2021-04-01T15:54:41Z DEBUG zenoh::net::routing::pubsub] Register subscription /@/router/94055E6C69404CC3B678B25E1B4C9B82/plugin/storages/backend/memory/storage/* for Face{0, 94055E6C69404CC3B678B25E1B4C9B82}
[2021-04-01T15:54:41Z DEBUG zenoh::net::routing::queries] Register queryable /@/router/94055E6C69404CC3B678B25E1B4C9B82/plugin/storages/backend/memory for Face{0, 94055E6C69404CC3B678B25E1B4C9B82}
[2021-04-01T15:54:41Z DEBUG zenoh::net::routing::resource] Register resource /@/router/94055E6C69404CC3B678B25E1B4C9B82/**
[2021-04-01T15:54:41Z DEBUG zenoh::net::routing::queries] Register queryable /@/router/94055E6C69404CC3B678B25E1B4C9B82/** for Face{0, 94055E6C69404CC3B678B25E1B4C9B82}
[2021-04-01T15:54:42Z DEBUG zplugin_dds] Local Domain Participant IH = 10277299952915539621
[2021-04-01T15:54:42Z DEBUG zplugin_dds] Discovery data from Participant with IH = 10277299952915539621
[2021-04-01T15:54:42Z DEBUG zplugin_dds] Discovered endpoint is keyless: false
[2021-04-01T15:54:42Z DEBUG zplugin_dds] Ignoring discovery from local participant: DCPSPublication
[2021-04-01T15:54:42Z DEBUG zplugin_dds] Local Domain Participant IH = 10277299952915539621
[2021-04-01T15:54:42Z DEBUG zplugin_dds] Discovery data from Participant with IH = 10277299952915539621
[2021-04-01T15:54:42Z DEBUG zplugin_dds] Discovered endpoint is keyless: false
[2021-04-01T15:54:42Z DEBUG zplugin_dds] Ignoring discovery from local participant: DCPSSubscription

Notebook:

docker run -it --rm --net=host --name zenoh -e TARGET_IP=10.212.20.70 -e MODE=peer -e CONNECTION_TYPE=e custom-zenoh-plugin-dds:latest

docker run -it --rm --net=host --name zenoh -e TARGET_IP=10.212.20.70 -e MODE=peer -e CONNECTION_TYPE=e custom-image:latest
1617292516.480436 [1]        dzd: using network interface wlo1 (udp/192.168.0.138) selected arbitrarily from: wlo1, docker0, tun0
[2021-04-01T15:55:16Z DEBUG zenoh::net] Zenoh Rust API v9d32433
[2021-04-01T15:55:16Z DEBUG zenoh::net] Config: local_routing=false;mode=peer;peer=tcp/10.212.20.70:7447;join_subscriptions=/demo/dds/**;join_publications=/demo/dds/**
[2021-04-01T15:55:16Z INFO  zenoh::net::runtime] Using PID: F6C7033CF20E440CB3CE959D8AEA0816
[2021-04-01T15:55:16Z DEBUG zenoh::net::routing::network] [Peers network] Add node (self) F6C7033CF20E440CB3CE959D8AEA0816
[2021-04-01T15:55:16Z DEBUG zenoh::net::runtime::orchestrator] Listener tcp/0.0.0.0:33475 added
[2021-04-01T15:55:16Z INFO  zenoh::net::runtime::orchestrator] zenohd can be reached on tcp/192.168.0.138:33475
[2021-04-01T15:55:16Z INFO  zenoh::net::runtime::orchestrator] zenohd can be reached on tcp/172.17.0.1:33475
[2021-04-01T15:55:16Z INFO  zenoh::net::runtime::orchestrator] zenohd can be reached on tcp/172.20.116.133:33475
[2021-04-01T15:55:16Z DEBUG zenoh::net::runtime::orchestrator] UDP port bound to 224.0.0.224:7447
[2021-04-01T15:55:16Z DEBUG zenoh::net::runtime::orchestrator] Joined multicast group 224.0.0.224
[2021-04-01T15:55:16Z INFO  zenoh::net::runtime::orchestrator] zenohd listening scout messages on 224.0.0.224:7447
[2021-04-01T15:55:16Z DEBUG zenoh::net::runtime::orchestrator] UDP port bound to 192.168.0.138:36664
[2021-04-01T15:55:16Z DEBUG zenoh::net::runtime::orchestrator] Waiting for UDP datagram...
[2021-04-01T15:55:16Z DEBUG zenoh::net::protocol::session::authenticator::shm] Peer 94055E6C69404CC3B678B25E1B4C9B82 did not express interest in shared memory
[2021-04-01T15:55:16Z DEBUG zenoh::net::protocol::session::manager] New session opened with 94055E6C69404CC3B678B25E1B4C9B82: whatami 1, sn resolution 268435456, initial sn tx 102609887, initial sn rx 213530380, is_local: false
[2021-04-01T15:55:16Z DEBUG zenoh::net::routing::network] [Peers network] Add node (link) 94055E6C69404CC3B678B25E1B4C9B82
[2021-04-01T15:55:16Z DEBUG zenoh::net::routing::router] New Face{0, 94055E6C69404CC3B678B25E1B4C9B82}
[2021-04-01T15:55:16Z DEBUG zenoh::net::runtime::orchestrator] Successfully connected to configured peer tcp/10.212.20.70:7447
[2021-04-01T15:55:16Z DEBUG zenoh::net::routing::network] [Peers network] Add node (state) 39A2C1DFCD3743B3823F7F0B84ECF585
[2021-04-01T15:55:16Z DEBUG zenoh::net::routing::router] New Face{1, F6C7033CF20E440CB3CE959D8AEA0816}
[2021-04-01T15:55:16Z DEBUG zenoh::net::routing::resource] Register resource /@/router/94055E6C69404CC3B678B25E1B4C9B82/plugin/storages/backend/memory/storage
[2021-04-01T15:55:16Z DEBUG zenoh::net::routing::network] Tree F6C7033CF20E440CB3CE959D8AEA0816 [None, Some("F6C7033CF20E440CB3CE959D8AEA0816 <- 94055E6C69404CC3B678B25E1B4C9B82"), Some("94055E6C69404CC3B678B25E1B4C9B82 <- 39A2C1DFCD3743B3823F7F0B84ECF585")]
[2021-04-01T15:55:16Z DEBUG zenoh::net::routing::network] Tree 94055E6C69404CC3B678B25E1B4C9B82 [Some("94055E6C69404CC3B678B25E1B4C9B82 <- F6C7033CF20E440CB3CE959D8AEA0816"), None, Some("94055E6C69404CC3B678B25E1B4C9B82 <- 39A2C1DFCD3743B3823F7F0B84ECF585")]
[2021-04-01T15:55:16Z DEBUG zenoh::net::routing::network] Tree 39A2C1DFCD3743B3823F7F0B84ECF585 [Some("94055E6C69404CC3B678B25E1B4C9B82 <- F6C7033CF20E440CB3CE959D8AEA0816"), Some("39A2C1DFCD3743B3823F7F0B84ECF585 <- 94055E6C69404CC3B678B25E1B4C9B82"), None]
[2021-04-01T15:55:16Z DEBUG zenoh::net::routing::resource] Register resource /@/router/94055E6C69404CC3B678B25E1B4C9B82/plugin/storages/backend/memory/storage/*
[2021-04-01T15:55:16Z DEBUG zenoh::net::routing::pubsub] Register peer subscription /@/router/94055E6C69404CC3B678B25E1B4C9B82/plugin/storages/backend/memory/storage/* (peer: 94055E6C69404CC3B678B25E1B4C9B82)
[2021-04-01T15:55:16Z DEBUG zenoh::net::routing::resource] Register resource /@/router/94055E6C69404CC3B678B25E1B4C9B82/plugin/storages/backend/*
[2021-04-01T15:55:16Z DEBUG zenoh::net::routing::pubsub] Register peer subscription /@/router/94055E6C69404CC3B678B25E1B4C9B82/plugin/storages/backend/* (peer: 94055E6C69404CC3B678B25E1B4C9B82)
[2021-04-01T15:55:16Z DEBUG zenoh::net::routing::queries] Register peer queryable /@/router/94055E6C69404CC3B678B25E1B4C9B82/plugin/storages/backend/memory (peer: 94055E6C69404CC3B678B25E1B4C9B82)
[2021-04-01T15:55:16Z DEBUG zenoh::net::routing::resource] Register resource /@/router/94055E6C69404CC3B678B25E1B4C9B82/**
[2021-04-01T15:55:16Z DEBUG zenoh::net::routing::queries] Register peer queryable /@/router/94055E6C69404CC3B678B25E1B4C9B82/** (peer: 94055E6C69404CC3B678B25E1B4C9B82)
[2021-04-01T15:55:17Z DEBUG zplugin_dds] Local Domain Participant IH = 13766400199328392408
[2021-04-01T15:55:17Z DEBUG zplugin_dds] Discovery data from Participant with IH = 13766400199328392408
[2021-04-01T15:55:17Z DEBUG zplugin_dds] Discovered endpoint is keyless: false
[2021-04-01T15:55:17Z DEBUG zplugin_dds] Ignoring discovery from local participant: DCPSPublication
[2021-04-01T15:55:17Z DEBUG zplugin_dds] Local Domain Participant IH = 13766400199328392408
[2021-04-01T15:55:17Z DEBUG zplugin_dds] Discovery data from Participant with IH = 13766400199328392408
[2021-04-01T15:55:17Z DEBUG zplugin_dds] Discovered endpoint is keyless: false
[2021-04-01T15:55:17Z DEBUG zplugin_dds] Ignoring discovery from local participant: DCPSSubscription
[2021-04-01T15:57:21Z DEBUG zenoh::net::routing::network] Remove node 39A2C1DFCD3743B3823F7F0B84ECF585
[2021-04-01T15:57:21Z DEBUG zenoh::net::routing::network] Tree F6C7033CF20E440CB3CE959D8AEA0816 [None, Some("F6C7033CF20E440CB3CE959D8AEA0816 <- 94055E6C69404CC3B678B25E1B4C9B82")]
[2021-04-01T15:57:21Z DEBUG zenoh::net::routing::network] Tree 94055E6C69404CC3B678B25E1B4C9B82 [Some("94055E6C69404CC3B678B25E1B4C9B82 <- F6C7033CF20E440CB3CE959D8AEA0816"), None]
[2021-04-01T15:57:24Z DEBUG zenoh::net::routing::network] [Peers network] Add node (state) D5D190BEA9C3440F8C39C150FE1E6A78
[2021-04-01T15:57:24Z DEBUG zenoh::net::routing::network] Tree F6C7033CF20E440CB3CE959D8AEA0816 [None, Some("F6C7033CF20E440CB3CE959D8AEA0816 <- 94055E6C69404CC3B678B25E1B4C9B82"), Some("94055E6C69404CC3B678B25E1B4C9B82 <- D5D190BEA9C3440F8C39C150FE1E6A78")]
[2021-04-01T15:57:24Z DEBUG zenoh::net::routing::network] Tree 94055E6C69404CC3B678B25E1B4C9B82 [Some("94055E6C69404CC3B678B25E1B4C9B82 <- F6C7033CF20E440CB3CE959D8AEA0816"), None, Some("94055E6C69404CC3B678B25E1B4C9B82 <- D5D190BEA9C3440F8C39C150FE1E6A78")]
[2021-04-01T15:57:24Z DEBUG zenoh::net::routing::network] Tree D5D190BEA9C3440F8C39C150FE1E6A78 [Some("94055E6C69404CC3B678B25E1B4C9B82 <- F6C7033CF20E440CB3CE959D8AEA0816"), Some("D5D190BEA9C3440F8C39C150FE1E6A78 <- 94055E6C69404CC3B678B25E1B4C9B82"), None]
[2021-04-01T15:59:23Z DEBUG zenoh::net::routing::network] Remove node D5D190BEA9C3440F8C39C150FE1E6A78
[2021-04-01T15:59:24Z DEBUG zenoh::net::routing::network] Tree F6C7033CF20E440CB3CE959D8AEA0816 [None, Some("F6C7033CF20E440CB3CE959D8AEA0816 <- 94055E6C69404CC3B678B25E1B4C9B82")]
[2021-04-01T15:59:24Z DEBUG zenoh::net::routing::network] Tree 94055E6C69404CC3B678B25E1B4C9B82 [Some("94055E6C69404CC3B678B25E1B4C9B82 <- F6C7033CF20E440CB3CE959D8AEA0816"), None]

Robot:

docker run -it --rm --net=host --name zenoh -e TARGET_IP=10.212.20.70 -e MODE=peer -e CONNECTION_TYPE=e custom-zenoh-plugin-dds:latest

docker run -it --rm --net=host --name zenoh -e TARGET_IP=10.212.20.70 -e MODE=peer -e CONNECTION_TYPE=e custom-image:latest
1617292635.854877 [1]        dzd: using network interface eno2 (udp/192.168.124.10) selected arbitrarily from: eno2, eno1, br-0c328cf4f120, docker0
[2021-04-01T15:57:15Z DEBUG zenoh::net] Zenoh Rust API v9d32433
[2021-04-01T15:57:15Z DEBUG zenoh::net] Config: mode=peer;local_routing=false;join_publications=/demo/dds/**;join_subscriptions=/demo/dds/**;peer=tcp/10.212.20.70:7447
[2021-04-01T15:57:15Z INFO  zenoh::net::runtime] Using PID: D5D190BEA9C3440F8C39C150FE1E6A78
[2021-04-01T15:57:15Z DEBUG zenoh::net::routing::network] [Peers network] Add node (self) D5D190BEA9C3440F8C39C150FE1E6A78
[2021-04-01T15:57:15Z DEBUG zenoh::net::runtime::orchestrator] Listener tcp/0.0.0.0:33019 added
[2021-04-01T15:57:15Z INFO  zenoh::net::runtime::orchestrator] zenohd can be reached on tcp/192.168.124.10:33019
[2021-04-01T15:57:15Z INFO  zenoh::net::runtime::orchestrator] zenohd can be reached on tcp/192.168.123.10:33019
[2021-04-01T15:57:15Z INFO  zenoh::net::runtime::orchestrator] zenohd can be reached on tcp/172.19.0.1:33019
[2021-04-01T15:57:15Z INFO  zenoh::net::runtime::orchestrator] zenohd can be reached on tcp/172.17.0.1:33019
[2021-04-01T15:57:15Z DEBUG zenoh::net::runtime::orchestrator] UDP port bound to 224.0.0.224:7447
[2021-04-01T15:57:15Z DEBUG zenoh::net::runtime::orchestrator] Joined multicast group 224.0.0.224
[2021-04-01T15:57:15Z INFO  zenoh::net::runtime::orchestrator] zenohd listening scout messages on 224.0.0.224:7447
[2021-04-01T15:57:15Z DEBUG zenoh::net::runtime::orchestrator] UDP port bound to 192.168.124.10:53993
[2021-04-01T15:57:15Z DEBUG zenoh::net::runtime::orchestrator] Waiting for UDP datagram...
[2021-04-01T15:57:15Z DEBUG zenoh::net::protocol::session::authenticator::shm] Peer 94055E6C69404CC3B678B25E1B4C9B82 did not express interest in shared memory
[2021-04-01T15:57:15Z DEBUG zenoh::net::protocol::session::manager] New session opened with 94055E6C69404CC3B678B25E1B4C9B82: whatami 1, sn resolution 268435456, initial sn tx 22189045, initial sn rx 35848901, is_local: false
[2021-04-01T15:57:15Z DEBUG zenoh::net::routing::network] [Peers network] Add node (link) 94055E6C69404CC3B678B25E1B4C9B82
[2021-04-01T15:57:15Z DEBUG zenoh::net::routing::router] New Face{0, 94055E6C69404CC3B678B25E1B4C9B82}
[2021-04-01T15:57:15Z DEBUG zenoh::net::runtime::orchestrator] Successfully connected to configured peer tcp/10.212.20.70:7447
[2021-04-01T15:57:15Z DEBUG zenoh::net::routing::network] [Peers network] Add node (state) F6C7033CF20E440CB3CE959D8AEA0816
[2021-04-01T15:57:15Z DEBUG zenoh::net::routing::network] Tree D5D190BEA9C3440F8C39C150FE1E6A78 [None, Some("D5D190BEA9C3440F8C39C150FE1E6A78 <- 94055E6C69404CC3B678B25E1B4C9B82"), Some("94055E6C69404CC3B678B25E1B4C9B82 <- F6C7033CF20E440CB3CE959D8AEA0816")]
[2021-04-01T15:57:15Z DEBUG zenoh::net::routing::network] Tree 94055E6C69404CC3B678B25E1B4C9B82 [Some("94055E6C69404CC3B678B25E1B4C9B82 <- D5D190BEA9C3440F8C39C150FE1E6A78"), None, Some("94055E6C69404CC3B678B25E1B4C9B82 <- F6C7033CF20E440CB3CE959D8AEA0816")]
[2021-04-01T15:57:15Z DEBUG zenoh::net::routing::network] Tree F6C7033CF20E440CB3CE959D8AEA0816 [Some("94055E6C69404CC3B678B25E1B4C9B82 <- D5D190BEA9C3440F8C39C150FE1E6A78"), Some("F6C7033CF20E440CB3CE959D8AEA0816 <- 94055E6C69404CC3B678B25E1B4C9B82"), None]
[2021-04-01T15:57:15Z DEBUG zenoh::net::routing::resource] Register resource /@/router/94055E6C69404CC3B678B25E1B4C9B82/plugin/storages/backend/memory/storage
[2021-04-01T15:57:15Z DEBUG zenoh::net::routing::resource] Register resource /@/router/94055E6C69404CC3B678B25E1B4C9B82/plugin/storages/backend/memory/storage/*
[2021-04-01T15:57:15Z DEBUG zenoh::net::routing::pubsub] Register peer subscription /@/router/94055E6C69404CC3B678B25E1B4C9B82/plugin/storages/backend/memory/storage/* (peer: 94055E6C69404CC3B678B25E1B4C9B82)
[2021-04-01T15:57:15Z DEBUG zenoh::net::routing::resource] Register resource /@/router/94055E6C69404CC3B678B25E1B4C9B82/plugin/storages/backend/*
[2021-04-01T15:57:15Z DEBUG zenoh::net::routing::pubsub] Register peer subscription /@/router/94055E6C69404CC3B678B25E1B4C9B82/plugin/storages/backend/* (peer: 94055E6C69404CC3B678B25E1B4C9B82)
[2021-04-01T15:57:15Z DEBUG zenoh::net::routing::queries] Register peer queryable /@/router/94055E6C69404CC3B678B25E1B4C9B82/plugin/storages/backend/memory (peer: 94055E6C69404CC3B678B25E1B4C9B82)
[2021-04-01T15:57:15Z DEBUG zenoh::net::routing::resource] Register resource /@/router/94055E6C69404CC3B678B25E1B4C9B82/**
[2021-04-01T15:57:15Z DEBUG zenoh::net::routing::queries] Register peer queryable /@/router/94055E6C69404CC3B678B25E1B4C9B82/** (peer: 94055E6C69404CC3B678B25E1B4C9B82)
[2021-04-01T15:57:16Z DEBUG zenoh::net::routing::router] New Face{1, D5D190BEA9C3440F8C39C150FE1E6A78}
[2021-04-01T15:57:16Z DEBUG zplugin_dds] Local Domain Participant IH = 4693299964885771766
[2021-04-01T15:57:16Z DEBUG zplugin_dds] Discovery data from Participant with IH = 4693299964885771766
[2021-04-01T15:57:16Z DEBUG zplugin_dds] Discovered endpoint is keyless: false
[2021-04-01T15:57:16Z DEBUG zplugin_dds] Ignoring discovery from local participant: DCPSPublication
[2021-04-01T15:57:16Z DEBUG zplugin_dds] Local Domain Participant IH = 4693299964885771766
[2021-04-01T15:57:16Z DEBUG zplugin_dds] Discovery data from Participant with IH = 4693299964885771766
[2021-04-01T15:57:16Z DEBUG zplugin_dds] Discovered endpoint is keyless: false
[2021-04-01T15:57:16Z DEBUG zplugin_dds] Ignoring discovery from local participant: DCPSSubscription

# Copyright (c) 2017, 2020 ADLINK Technology Inc.
#
# This program and the accompanying materials are made available under the
# terms of the Eclipse Public License 2.0 which is available at
# http://www.eclipse.org/legal/epl-2.0, or the Apache License, Version 2.0
# which is available at https://www.apache.org/licenses/LICENSE-2.0.
#
# SPDX-License-Identifier: EPL-2.0 OR Apache-2.0
#
# Contributors:
#   ADLINK zenoh team, <[email protected]>

FROM ros-image as builder # i replaced this

WORKDIR /usr/src

# Required for correct installation of maven package
RUN mkdir -p /usr/share/man/man1/

RUN apt-get update && apt-get -y install g++ cmake libssl-dev openjdk-11-jdk-headless maven git clang

RUN git clone https://github.com/eclipse-zenoh/zenoh-plugin-dds.git
WORKDIR /usr/src/zenoh-plugin-dds

RUN ./config.sh
ENV RUSTUP_HOME=/usr/local/rustup \
    CARGO_HOME=/usr/local/cargo \
    PATH=/usr/local/cargo/bin:$PATH

RUN set -eux; \
    \
    apt-get install wget; \
    url="https://static.rust-lang.org/rustup/dist/x86_64-unknown-linux-gnu/rustup-init"; \
    wget "$url"; \
    chmod +x rustup-init; \
    ./rustup-init -y --no-modify-path --default-toolchain nightly; \
    rm rustup-init; \
    chmod -R a+w $RUSTUP_HOME $CARGO_HOME; \
    rustup --version; \
    cargo --version; \
    rustc --version;
    

RUN cargo +nightly build --release --all-targets
RUN cargo install --path . --locked

###
### Run part
###
FROM ros-image

RUN apt-get update && apt-get install -y libssl1.1 && rm -rf /var/lib/apt/lists/*
COPY --from=builder /usr/local/lib/libdds* /usr/local/lib/libcdds-util.so /usr/local/lib/
COPY --from=builder /usr/local/cargo/bin/dzd /usr/local/bin/dzd
COPY --from=builder /usr/src/zenoh-plugin-dds /usr/src/zenoh-plugin-dds
RUN ldconfig -v

ENV CYCLONE_INCLUDE=/opt/ros/foxy/include
ENV CYCLONE_LIB=/opt/ros/foxy/lib/x86_64-linux-gnu
        
WORKDIR /usr/src/zenoh-plugin-dds
        
ENV TARGET_IP 10.212.11.176
ENV CONNECTION_TYPE l
ENV MODE peer
ENV DOMAIN 1

EXPOSE 7447/udp
EXPOSE 7447/tcp

CMD bash -c "RUST_LOG=debug dzd --scope /demo/dds -r /demo/dds/** -w /demo/dds/** -m $MODE -d $DOMAIN -$CONNECTION_TYPE tcp/$TARGET_IP:7447"

Reproduced on rust-master branch (21b346a):

• start zenohd
• curl http://localhost:8000/@/router/localreturns no key/value
• while curl http://localhost:8000/@/router/<zenoh-id> returns a JSON object with status information on the zenoh router

The first generation zenoh infrastructure was written in OCaml. We had selected OCaml as we wanted to have the benefits of a high-level, secure and performant programming language. Whilst we are very happy in general with OCaml, a few points have lead us to consider a Rust rewrite:

• In spite of OCaml beauty and effectiveness, there are few programmers that are proficient. This limits somehow the contribution.
• Some of our end-users have raised concerns with respect to their ability to maintain an OCaml based code base.

Rust, was introduced as a system programming language that promised zero-cost abstractions. It runs with a C-equivalent runtime but provides some very nice abstractions. Additionally Rust integrates quite nicely with the C/C++ ecosystem. Thus for us Rust is the right language for the next generation of zenoh.

Below is the status of Rust re-write compared to the OCaml version:

• network protocol re-design
• peer-to-peer mode (was not implemented in OCaml)
• zenoh-net API in Rust
• zenoh-net examples in Rust, including throughput test
• zenohd broker (without routing capabilities)
• zenohd router (with routing capabilities)
• new zenoh-c API based on zenoh-net in Rust
• admin space
• plugins loading mechanism
• REST plugin
• Storages plugin
• Memory backend
• InfluxDB backend
• zenoh API in Rust
• zenoh-python API update
• zenoh-java API update (relying on the new zenoh-c API)
• zenoh-go API update (relying on the new zenoh-c API)

Ideally we would want to avoid any copy when sending large data, such as images or point clouds, or anything else sufficiently big.

zenoh should provide a way to allocate and manage shared memory buffers and transparently transfer them using zero-copy to subscribers on the same node while streaming the content over sockets for subscribers across the network.

All of this should be as transparent as possible to the users and the memory management for the shared-memory buffers should follow Rust ownership (as much as possible).

Hi,

Debian package generation is not working.

Running cargo deb -p zenoh gives:

$ cargo deb -p zenoh
warning: extended-description field missing. Using ../README.md, but markdown may not render well.
cargo-deb: unable to read extended description from file: ../README.md
  because: No such file or directory (os error 2)

Instead is should build and generate the debian package.

Add a backend for file system that will:

• allow creation of storages, each mapping to a directory on the local file system. Each storage can be configured as read/write or read-only.
• store each zenoh path/value publication as a file where:
  • each zenoh path is mapped to a file system path. the zenoh path will be stripped from a configurable "path_prefix", and the remaining will be used as relative path from the storage's directory.
• each value will correspond to the content of the file
• store the zenoh meta-data (timestamp, encoding, deleted flag) in a side file (1 per directory), that will not be exposed in the zenoh storage. For pre-existing files (not created via zenoh), the timestamp will be generated from the file's modification time and the encoding will be deducted from the file's extension.

We should provide a C and ideally a C++ API for our Rust implementation of zenoh. We are currently working on a C API, C++ will come next.

When using the UnixDomain Socket as transport in a Zenoh router, and killing the router with Ctrl-C the .sock file is not removed, hence when restarting zenohd crashes raising an error.

[2021-01-19T11:34:35Z WARN  zenoh_protocol::link::unixsock_stream] Can not create a new UnixSock-Stream listener on "/tmp/zenoh.sock": Address already in use (os error 98)
[2021-01-19T11:34:35Z ERROR zenoh_router::runtime::orchestrator] Unable to open listener unixsock-stream//tmp/zenoh.sock : Invalid link (Can not create a new UnixSock-Stream listener on "/tmp/zenoh.sock": Address already in use (os error 98)) at zenoh-protocol/src/link/unixsock_stream.rs:760.
IO error () at zenoh-router/src/runtime/orchestrator.rs:226. - Caused by Invalid link (Can not create a new UnixSock-Stream listener on "/tmp/zenoh.sock": Address already in use (os error 98)) at zenoh-protocol/src/link/unixsock_stream.rs:760.. Exiting...

This seems related to the implementation of the UnixDomain Socket transport, it should create a lock file alongside the socket file, following this approach when it is time to "create" the file.

I've broken up the diff per package so that they are easier to review and they can be put on hold if someone is actively working on any of them.

• plugins/example-plugin #25
• plugins/zenoh-http #26
• zenoh #27
• zenoh-util #28
• zenoh-router #29
• zenoh-protocol #30
• zenoh-perf #31
• zenoh-ffi #32

Once all the PR are merged, we can merge this one that will run rustfmt on CI:

• Run rustfmt as part of CI #24

It would be desirable to have a zenoh-net API to receive notifications about new sessions being established and potentially sessions being closed. The properties of the session should be passed to the open notification and minimally the PeerId to the close.

When routing data, the router still holds the tables write lock while calling the primitives of the outgoing faces. When such a face is a local API, it prevents the called code to call back operations on the router primitives in the same task (otherwise leads to a deadlock).
This prevents zenoh-net users to call operations like write() or query() in callback_subscribers. It even lead to deadlocks when using stream based subscribers on high frequency.

The zenoh API reports a misleading info log at start-up.
For instance running zn_info example:

[2021-05-26T16:42:56Z INFO  zenoh::net::runtime::orchestrator] zenohd can be reached on tcp/192.168.1.22:54654

Looks like https://docs.rs/cxx/0.4.7/cxx/ would be a good approach in terms of safety and performance to call Rust code from C++.

@kydos Could you define the C++ APIs that we need?

Hi,
Is it possible to have access control to keys? An ability to control the visibility and write access may be useful in some scenarios.

Regards,
Sojan

The target parameter of the zenoh-net query() function should be interpreted and used by the infrastructure to determine which queryables, amongst all the matching ones in the system, should be used to answer the query.

The zenoh is quite a giant beast to work on. Let me share some ideas that would improve the developing experience.

Collect std and external imports to one common.rs

I used to have a top level common.rs like this.

pub use std::{
    path::{Path, PathBuf}.
    // ...
};
pub use itertools::Itertools;
//...

and loads the common imports when needed.

use crate::common::*;

Add rustfmt.toml

I prefer the rustfmt.toml to keep the code tidy.

edition = "2018"  # required by rust-analyzer
merge_imports = true
format_code_in_doc_comments = true

In-file mods

This is a little trick that I found useful to organize the code. For whatever a file with more than one types and traits, group them into private mod and export them.

pub use workspace::*;
pub use data::*;

mod workspace {
    use super::*;
    // structs, impls, .., etc
}

mod data {
    use super::*;
    // structs, impls, .., etc
}

Most of them require non-trivial changes. If you are comfortable with them, I can push a PR for this.

CI Passing badge should click through to CI

It would be good to be able to deploy several routers at the edge between a peers sub-network and a network of routers both for load balancing and fault tolerance purposes.

Rational

We need to replace the usage of # as separator for fragments in Selector because most of the web client won't send the fragment part of the URL to the zenoh HTTP plugin.

Detail

The current (in 0.5.0-beta.5) specification of a zenoh Selector is the following:

/s1/s2/.../sn?x>1&y<2&...&z=4(p1=v1;p2=v2;...;pn=vn)#a;b;x;y;...;z
|           | |             | |                   |  |           |
|-- expr ---| |--- filter --| |---- properties ---|  |--fragment-|

where:

• expr: is a Path Expression (e.g.: /demo/example/**)
• filter: a list of predicates separated by '&' allowing to perform filtering on the values
  associated with the matching keys. Each predicate has the form "field-operator-value" value where:
  • field is the name of a field in the value (is applicable and is existing. otherwise the predicate is false)
  • operator is one of a comparison operators: < , > , <= , >= , = , !=
  • value is the the value to compare the field’s value with
• fragment: a list of fields names allowing to return a sub-part of each value. A.k.a. a projection in the mathematical sense of the term.
  This feature only applies to structured values using a “self-describing” encoding, such as JSON or XML.
  It allows to select only some fields within the structure. A new structure with only the selected fields
  will be used in place of the original value.

For instance, if we imagine GPS coordinates of objects stored zenoh's time-series storages, under an undetermined hierarchy prefixed by /demo/example/, this Selector:
/demo/examples/**/GPS-coordinate?lat>0(starttime=now()-1h)#lat;long;name
will retrieve all the GPS data of objects in the north hemisphere (lat>0) within the last hour (starttime=now()-1h) and for each data only the latitude, the longitude and the name should be returned.

A Selector can easily be used to get path/values from zenoh via the HTTP plugin, simply prepending the selector with the base URL of a zenoh router. E.g.:
http://localhost:8000/demo/examples/**/GPS-coordinate?lat>0(starttime=now()-1h)#lat;long;name
However, the majority of HTTP clients (web browsers, curl...) never send the fragment part of the URL (starting from #) to the web server (the zenoh HTTP plugin in our case). As a consequence, a web client won't be able to request only sub-parts of data.

When trying to run the example on a raspberry pi (docker), I get an Exec Format error.

ubuntu@ubuntu:~$ docker run --init -p 7447:7447/tcp -p 7447:7447/udp -p 8000:8000/tcp eclipse/zenoh
[FATAL tini (6)] exec /entrypoint.sh failed: Exec format error

In the past when I have seen this, it is typically due to lack of arm support for the image. Is there an arm dockerhub image available?

Potential security breach. See /etc/passwd

Hello! I discovered Zenoh days ago and I have some questions about it.

I want to use Zenoh in a nRF BLE boards, but I really don't now how to do it. I have to compile Zenoh and use it as a simple library or I have to integrated in some RTO like Zephyr...?

Also, Zenoh works over BLE or it is needed a custom layer to have IP over BLE?

Thanks!

Using a fresh and empty InfluxDB server, it may happens that the zenoh router raises the following exception when adding a storage using the Influxdb backend:

[1587746557.085046][INFO] Storage listener raised exception (Failure Influxdb_types.serie.values) :
Raised at file "src/zenoh-storages/zenoh-storages-be/zenoh-storages-be-influxdb/influxdb_types.ml", line 48, characters 0-79
Called from file "src/zenoh-storages/zenoh-storages-be/zenoh-storages-be-influxdb/influxdb_driver.ml", line 40, characters 22-78
Called from file "src/core/lwt.ml", line 1930, characters 23-26
Re-raised at file "src/zenoh-storages/zenoh-storages-be/zenoh-storages-be-influxdb/influxdb_driver.ml", line 35, characters 2-727
Re-raised at file "src/zenoh-storages/zenoh-storages-be/zenoh-storages-be-influxdb/influxdb_driver.ml", line 187, characters 2-62
Re-raised at file "src/zenoh-storages/zenoh-storages-be/zenoh-storages-be-influxdb/zenoh_storages_be_influxdb.ml", line 134, characters 6-542
Re-raised at file "src/zenoh-storages/zenoh_storages.ml", line 196, characters 4-741

Hi,

I have a deserialization error when deserializing data received via a SharedMemoryBuf.
Seems that data is modified by the publisher while the subscriber is reading it.

Currently (v0.5.0-beta.8) the zenoh plugins are dynamic libraries that are loaded by zenohd at startup.
They just must implement 2 #[no_mangle] functions:

// return the list of command lines arguments expected by the plugin
fn get_expected_args<'a, 'b>() -> Vec<clap::Arg<'a, 'b>>;

// start the plugin with command line arguments
fn start(runtime: Runtime, args: clap::ArgMatches<'static>);

Now we would like to be able to use the zenoh plugins in any zenoh Rust application, but in a static way (i.e. just declaring the plugins in the application's cargo.toml and calling them explicitly in the main() operation).

Use case:
The zenoh-dds-plugin can be built as a standalone zenoh application named zenoh-bridge-dds.
We would like to administrate it at runtime in the same way as zenohd: via an admin space accessible through the REST API. The REST API is implemented as a zenoh plugin. But we don't want the zenoh-bridge-dds to have the ability to dynamically load any plugin. The solution is to statically link zenoh-bridge-dds with the zenoh-rest plugin.
And as the code of the DDS/zenoh bridge itself should eventually be also a zenoh plugin (to be loadable by zenohd), the zenoh-bridge-dds will finally just by a main program starting the 2 plugins that are linked in static.

Cannot compile rust-master Zenoh code due to crates dependencies on a new git clone. Some of the .toml files have outdated dependencies. For example, async-sse is still on v3.0.0, while the latest version is v4.0.1. Currently async-sse v3.0.0 won't compile.

If you update the tide to latest 0.13.0 (which will use latest async-sse v4.0.1), then plugins/zenoh-http/src/lib.rs won't compile.

$ cargo outdated
zenoh

Name Project Compat Latest Kind Platform

async-std 1.6.4 --- 1.6.5 Normal ---
async-task 3.0.0 --- Removed Normal ---

zenoh-protocol

Name Project Compat Latest Kind Platform

async-std 1.6.4 --- 1.6.5 Normal ---
async-task 3.0.0 --- Removed Normal ---

zenoh-util

Name Project Compat Latest Kind Platform

async-std 1.6.4 --- 1.6.5 Normal ---
async-task 3.0.0 --- Removed Normal ---

zenoh-router

Name Project Compat Latest Kind Platform

async-std 1.6.4 --- 1.6.5 Normal ---
async-task 3.0.0 --- Removed Normal ---

zplugin-example

Name Project Compat Latest Kind Platform

async-std 1.6.4 --- 1.6.5 Normal ---
async-task 3.0.0 --- Removed Normal ---

zplugin-http

Name Project Compat Latest Kind Platform

async-sse 3.0.0 --- 4.0.1 Normal ---
async-std 1.6.4 --- 1.6.5 Normal ---
async-task 3.0.0 --- Removed Normal ---
route-recognizer 0.1.13 --- 0.2.0 Normal ---
tide 0.11.0 --- 0.13.0 Normal ---

zplugin_storages

Name Project Compat Latest Kind Platform

async-std 1.6.4 --- 1.6.5 Normal ---
async-task 3.0.0 --- Removed Normal ---

The build should work for ubuntu-latest and macos-latest, and make available artifacts for download.

I'm wondering how that would be possible. Workspace holds a Zenoh instance, so just locking Zenoh with a Mutex (e.g. the tokio one, which is Sendable) would not be safe. Is this even supported to begin with or am I just using the library wrong?

Ideally I would like to just get a bunch of ChangeStreams and use those around in my application, but how can I pass them around, when they require a Workspace instance with a longer lifetime?

Make available a zenoh C client library for micro-controllers.

zenoh should be able to operate natively over Bluetooth.

The current implementation of zenoh shared memory allocators retrieves the memory for dealing with free list on the shared memory segment. This decision was taken at the time when we considered that a reference dropping to zero would cause a chunk to go back to the free list. However we opted to go for mark and sweep garbage collection that does not require processes to coordinate while operating on the free-list. A a consequence only the process owing the shared-memory segment will do the garbage collection and thus all administration data can be kept on the heap as opposed to the shared memory.
Thus we should simplify the implementation by using plain rust containers to keep shared memory bookkeeping and allocate these on the heap. This will not only simplify the code but also avoid using shared memory for administration and devoting it completely to user data.

Zenoh is currently able to deal with string, json, sql, properties and raw types (bytes array). It would be extremely convenient for IoT applications that use resources to represent digital or analog sensors to support natively int and floats

There are many great observability tools available from Cloud Native ecosystem, such as prometheus, grafana, etc. Cloud Native/CNCF has proposed a new telemetry specification: open-telemetry, which is a collection of tools, APIs, and SDKs. You use it to instrument, generate, collect, and export telemetry data (metrics, logs, and traces) for analysis in order to understand your software's performance and behavior.

Zenoh as industrial grade software will need the (standard) observability leveraging cloud-native eco-systems. Zenoh team need to put this in the roadmap, and have a plan to implement this:

https://opentelemetry.io/
https://github.com/open-telemetry

There is a RUST open telemetry alpha library:
https://crates.io/crates/opentelemetry

Following discussion stated in #82 (comment)

At the moment, to keep track of the incoming uninitialized sessions we use a Arc<Mutex<HashSet<Link>>>.

I suggest we use the hashes of the Cookies instead of the Link. That way a peer cannot try to open multiple sessions after performing only one "session initialization".

ubuntu@ecs-zenoh-yhe-01:~/zenoh/target/release$ ./zenohd --version
The zenoh router v0.5.0-beta.5-224-g87cf763-modified built with rustc 1.51.0-nightly (2987785df 2020-12-28)

Start the first zenohd instance with RUST_LOG=debug flag, no issue:
[2021-02-19T15:14:33Z DEBUG zenoh_router::routing::pubsub] Register subscription /@/router/3C24E4AEED654EC48C521E191957EB19/plugin/storages/backend/* for face 0
[2021-02-19T15:14:33Z DEBUG zenoh_router::routing::pubsub] Register router subscription /@/router/3C24E4AEED654EC48C521E191957EB19/plugin/storages/backend/* (router: 3C24E4AEED654EC48C521E191957EB19)
[2021-02-19T15:14:33Z DEBUG zenoh_router::routing::pubsub] Register peer subscription /@/router/3C24E4AEED654EC48C521E191957EB19/plugin/storages/backend/* (peer: 3C24E4AEED654EC48C521E191957EB19)
[2021-02-19T15:14:33Z DEBUG zenoh_router::routing::router] New face 2
[2021-02-19T15:14:33Z INFO tide::server] Server listening on http://0.0.0.0:8000

Then start the second zenohd instance with RUST_LOG=debug flag on different machine on the same subnet, core dump:

ubuntu@ecs-zenoh-yhe-01:/zenoh/target/release$ RUST_LOG=debug ./zenohd
[2021-02-19T15:15:00Z DEBUG zenohd] zenohd v0.5.0-beta.5-224-g87cf763-modified built with rustc 1.51.0-nightly (2987785df 2020-12-28)
[2021-02-19T15:15:00Z DEBUG zenoh_router::plugins] Plugins to load: []
[2021-02-19T15:15:00Z DEBUG zenoh_util::lib_loader] Search for libraries libzplugin_*.so to load in ["/usr/local/lib", "/usr/lib", "/home/ubuntu/.zenoh/lib", "/home/ubuntu/zenoh/target/release", "/home/ubuntu/zenoh/target/release"]
[2021-02-19T15:15:00Z DEBUG zenoh_util::lib_loader] Do not load plugin storages from "/home/ubuntu/zenoh/target/release/libzplugin_storages.so" : already loaded.
[2021-02-19T15:15:00Z DEBUG zenoh_util::lib_loader] Do not load plugin rest from "/home/ubuntu/zenoh/target/release/libzplugin_rest.so" : already loaded.
[2021-02-19T15:15:00Z DEBUG zenoh_util::lib_loader] Do not load plugin storages from "/home/ubuntu/zenoh/target/release/libzplugin_storages.so" : already loaded.
[2021-02-19T15:15:00Z DEBUG zenoh_router::plugins] Plugin storages loaded from /usr/lib/libzplugin_storages.so
[2021-02-19T15:15:00Z DEBUG zenoh_router::plugins] Plugin rest loaded from /home/ubuntu/zenoh/target/release/libzplugin_rest.so
[2021-02-19T15:15:00Z DEBUG zenohd] Config: {"multicast_scouting": "true", "peer": "", "listener": "tcp/0.0.0.0:7447", "mode": "router", "add_timestamp": "true"}
[2021-02-19T15:15:00Z INFO zenoh_router::runtime] Using PID: E354454D56274CC4B6EB457750C2D651
[2021-02-19T15:15:00Z DEBUG zenoh_router::routing::network] [Routers network] Add node (self) E354454D56274CC4B6EB457750C2D651
[2021-02-19T15:15:00Z DEBUG zenoh_router::routing::network] [Peers network] Add node (self) E354454D56274CC4B6EB457750C2D651
[2021-02-19T15:15:00Z DEBUG zenoh_router::runtime::orchestrator] Listener tcp/0.0.0.0:7447 added
[2021-02-19T15:15:00Z INFO zenoh_router::runtime::orchestrator] zenohd can be reached on tcp/10.1.101.216:7447
[2021-02-19T15:15:00Z INFO zenoh_router::runtime::orchestrator] zenohd can be reached on tcp/172.17.0.1:7447
[2021-02-19T15:15:00Z DEBUG zenoh_router::runtime::orchestrator] UDP port bound to 224.0.0.224:7447
[2021-02-19T15:15:00Z DEBUG zenoh_router::runtime::orchestrator] Joined multicast group 224.0.0.224
[2021-02-19T15:15:00Z INFO zenoh_router::runtime::orchestrator] zenohd listening scout messages on 224.0.0.224:7447
[2021-02-19T15:15:00Z DEBUG zenoh_router::runtime::orchestrator] UDP port bound to 10.1.101.216:44347
[2021-02-19T15:15:00Z DEBUG zenoh_router::plugins] Start plugin storages
[2021-02-19T15:15:00Z DEBUG zenoh_router::runtime::orchestrator] Waiting for UDP datagram...
[2021-02-19T15:15:00Z DEBUG zenoh_router::plugins] Start plugin rest
thread 'async-std/runtime' panicked at 'range end index 94126439026992 out of range for slice of length 16', zenoh-protocol/src/core/mod.rs:184:10
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace
Aborted (core dumped)
ubuntu@ecs-zenoh-yhe-01:/zenoh/target/release$

SQLx managed to avoid locking themselves to tokio/async-std/smol or whatever seemingly by creating a small feature flag driven crate of about 200 loc.

It would be nice to avoid locking this directly to async-std, even if there is a compatibility layer.

https://github.com/launchbadge/sqlx/blob/master/sqlx-rt/src/lib.rs

QUIC protocol should be added to the list of transport protocols supported by zenoh. QUIC is a new protocol providing lower latency than TCP and integrate at its core security concepts for confidentiality and authentication.

1. Query messages include a consolidation field which is always sent on the wire. However, this field may not be relevant to the final recipient of the query message, i.e., the target of the query.
2. ReplyContext requires to always send the source_info field even in the case it is a final reply. However, this field is not relevant in such a case.
3. ReplyContext has source_kind and replier_id fields. But both source and replier refer to the same concept.

Therefore, three improvements could be made in the message structure:

1. Make the consolidation field optional in the Query message;
2. Make the source_info field optional in the ReplyContext decorator;
3. Rename source_info and/or replier_id fields to have coherent names;

The goal is to have the Jenkins CI build (https://ci.eclipse.org/zenoh/job/zenoh-releases) to build and push a Docker image of zenohd in https://hub.docker.com/r/eclipse/zenoh