Giter VIP home page Giter VIP logo

atm0s-sdn's Introduction

License: MIT Discord

atm0s-sdn: Global-scaled Ultra-low latency Decentralized SDN

A SAN I/O driven, open-source decentralized network infrastructure that can deliver high-quality data with minimal latency and efficient cost, similar to what Cloudflare achieves for their network.

For a deep dive into the technical aspects of network architecture, please refer to our Smart-Routing

Project Status: Alpha

This project is currently in the alpha stage of development. It is actively being developed and may undergo significant changes, including changes to the API. The primary focus at this stage is to make the project stable and reliable.

We are working with some potential partners to integrate the network in real-world scenarios. If you are interested in testing the network, feel free to contact us or sending an issues.

Features

  • High availability by being fully distributed, with no central controller.
  • Multi-zone support, high scalability.
  • Metric based Adaptive routing: latency, bandwidth
  • Network orchestration and discovery (manual mode only).
  • High extendibility by using Network Service.
  • Built-in features: PubSub, KeyValue ..
  • Cross platform: Linux, MacOs.

Build-in features

  • Router sync: automatically sync routing table between nodes for implementing Smart-Routing
  • Pubsub: Publish/Subscribe
  • DHT Multi-Map: Key-Value store
  • Node Alias: Each node can have multiple alias
  • Virtual Socket: Act as virtual UDP socket
  • Network accelerator by eBPF redirect
  • Authentication and Encryption

Build-in services

  • Visualization: show network structure
  • Manual discovery: each node can set owner tags and interested tags. each node will trying to connect to other nodes that have the interested tags.

Architecture

We split logic in to 2 parts: feature and service.

  • Feature: is a high-level logic that can be used to build a network service. For example, PubSub, KeyValue, Virtual Socket ..
  • Service: is a extendable logic that can be used to build a custom logic. Each service can use all other features to build a custom logic.

Network Protocol

For better in testing and developing we split library to 2 parts:

  • Network Protocol: implement in SANS-I/O style
  • Runner: for integrating with SANS-I/O runtime

Getting started

Project includes some example and a standalone node, which can be used to test the network.

Build from source:

cd bin
cargo build --release

Running first seed node as a network structure collector:

cargo run -- --collector --local-tags demo --connect-tags demo --node-id 1 --bind-addr 127.0.0.1:10001 --web-addr 0.0.0.0:3000

Running second nodes and join to network with seed node (you need to replace with seed node IP if it running on another device):

cargo run -- --local-tags demo --connect-tags mode --seeds 1@/ip4/127.0.0.1/udp/10001 --node-id 2 --bind-addr 127.0.0.1:10002

Same with this, we can run more nodes and connect to the network. Remember change node-id and port for not conflict with other nodes.

Access to the web interface to see the network structure: http://localhost:3000

Network Structure

You can also enable vpn feature in each node by add --vpn flag. After that, each node will be assigned with a private with rule: 10.33.33.{node_id % 8}.

Benchmarks

Network optimizer

We run 9 nodes across Asia, Us, Europe with Digital Ocean and some local ISP in Vietnam and test the latency between each node. The result show that the network can speed up latency by maximum 48.5% compare to normal routing.

Maximum optimized:

US EU ASIA
US 0
EU 3.40% 0
ASIA 48.5% 28.30% 47%

Average optimized:

US EU ASIA
US 0
EU 0.40% 0
ASIA 11.6% 9.50% 3%

We will test more in the future to get more accurate result with more cloud provider and local ISP.

Speeds

In some fast test, it show that network node can relay around 1Gbps per core. We will test more in the future to get more accurate result.

Showcases

  • Decentralized Media Server: Repo
  • Decentralized VPN: Repo
  • Decentralized Reverse Proxy: Repo

Contributing

The project is continuously being improved and updated. We are always looking for ways to make it better, whether that's through optimizing performance, adding new features, or fixing bugs. We welcome contributions from the community and are always looking for new ideas and suggestions. If you find it interesting or believe it could be helpful, we welcome your contributions to the codebase or consider starring the repository to show your support and motivate our team!

For more information, you can join our Discord channel

Roadmap

The first version will be released together with Media Server at the end of 2023.

Details on our roadmap can be seen TBA.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments

We would like to thank all the contributors who have helped in making this project successful.

atm0s-sdn's People

Contributors

binhduong85 avatar dhilipsiva avatar giangndm avatar giangndm-bluesea avatar github-actions[bot] avatar gramidt avatar luongngocminh avatar renovate[bot] avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

atm0s-sdn's Issues

Migration - moving logic out of I/O

Refactor with Separated Logic model:

  • Each logic will be composed inside a sync logic box, with multi inputs and outputs line
  • Logic box don't hold timer, instead it receive timestamp from input, or tick event
while let Some(event) = io.recv().wait {
  logic.on_event(event)?;
  while let Some(action) = logic.pop_action() {
      //process action here
  }
}

With above model, we can easy testing logic without depend on timer or something like async or I/O

Feature Request: Support multiple bind address

We need to support bind to multi address like:

  • 127.0.0.1:10000
  • 192.168.0.100:10000
  • 123.123.123.123:10000

This allow each node have chance to finding most optimal way to reduce cost in some cloud provide. We will have some priority with connection type: local > LAN > public internet

PubSub: allow subscribe with max desired layer info

In some applications like media-server, each channel can be split to multiple layers like video SVC or Simulcast. Therefore if we can config max desired layer or max desired bitrate, we can reduce waste network transferring

Update manual-discovery with node-tags

A node can be started with some tags, and setting up to connect to all node which has some tags.
It is useful when we create structured network for an application like streaming cloud services.
Note that, we don't want to using any centralized server

Wrong local pubsub consumer release

When multi times consumer create/destroy cause warning about Consumer not found

[2023-11-03T04:12:49.112Z INFO  pub_sub::behaviour] [PubSubServiceBehaviour 1] on added local channel 4086742429 => set Hashmap field
[2023-11-03T04:12:49.128Z INFO  pub_sub::behaviour] [PubSubServiceBehaviour 1] will sub hashmap 4086742429
[2023-11-03T04:12:49.128Z DEBUG pub_sub::behaviour] [PubSubServiceBehaviour 1] channel 4086742429 add source 1
[2023-11-03T04:12:49.128Z INFO  pub_sub::behaviour] [PubSubServiceBehaviour 1] on channel 4086742429 added source 1
[2023-11-03T04:12:49.128Z DEBUG pub_sub::relay] [PubsubRelay] channel 4086742429 added source  1 => auto sub for local subs [0]
[2023-11-03T04:12:49.128Z INFO  pub_sub::relay::logic] [PubsubRelayLogic 1] local sub 4086742429/1 event from 0 pushed to list, new relay
[2023-11-03T04:12:49.153Z INFO  pub_sub::behaviour] [PubSubServiceBehaviour 1] on added local channel 2861764290 => set Hashmap field
[2023-11-03T04:12:49.155Z INFO  pub_sub::behaviour] [PubSubServiceBehaviour 1] will sub hashmap 2861764290
[2023-11-03T04:12:49.155Z DEBUG pub_sub::behaviour] [PubSubServiceBehaviour 1] channel 2861764290 add source 1
[2023-11-03T04:12:49.155Z INFO  pub_sub::behaviour] [PubSubServiceBehaviour 1] on channel 2861764290 added source 1
[2023-11-03T04:12:49.155Z DEBUG pub_sub::relay] [PubsubRelay] channel 2861764290 added source  1 => auto sub for local subs [1]
[2023-11-03T04:12:49.155Z INFO  pub_sub::relay::logic] [PubsubRelayLogic 1] local sub 2861764290/1 event from 1 pushed to list, new relay






[2023-11-03T04:12:53.465Z INFO  pub_sub::behaviour] [PubSubServiceBehaviour 1] on removed local channel 2861764290 => del Hashmap field
[2023-11-03T04:12:53.465Z INFO  pub_sub::behaviour] [PubSubServiceBehaviour 1] on removed local channel 4086742429 => del Hashmap field
[2023-11-03T04:12:53.471Z DEBUG pub_sub::behaviour] [PubSubServiceBehaviour 1] channel 2861764290 remove source 1
[2023-11-03T04:12:53.471Z DEBUG pub_sub::behaviour] [PubSubServiceBehaviour 1] channel 4086742429 remove source 1
[2023-11-03T04:12:53.471Z INFO  pub_sub::behaviour] [PubSubServiceBehaviour 1] on channel 2861764290 removed source 1
[2023-11-03T04:12:53.471Z DEBUG pub_sub::relay] [PubsubRelay] channel 2861764290 removed source 1 => auto unsub for local subs [1]
[2023-11-03T04:12:53.471Z INFO  pub_sub::relay::logic] [PubsubRelayLogic 1] local unsub 2861764290/1 event from 1 removed from list
[2023-11-03T04:12:53.471Z INFO  pub_sub::relay::logic] [PubsubRelayLogic 1] local unsub 2861764290/1 event from 1 list empty in source node => removed
[2023-11-03T04:12:53.471Z INFO  pub_sub::behaviour] [PubSubServiceBehaviour 1] on channel 4086742429 removed source 1
[2023-11-03T04:12:53.471Z DEBUG pub_sub::relay] [PubsubRelay] channel 4086742429 removed source 1 => auto unsub for local subs [0]
[2023-11-03T04:12:53.471Z INFO  pub_sub::relay::logic] [PubsubRelayLogic 1] local unsub 4086742429/1 event from 0 removed from list
[2023-11-03T04:12:53.471Z INFO  pub_sub::relay::logic] [PubsubRelayLogic 1] local unsub 4086742429/1 event from 0 list empty in source node => removed






[2023-11-03T04:12:57.650Z INFO  pub_sub::behaviour] [PubSubServiceBehaviour 1] on added local channel 4086742429 => set Hashmap field
[2023-11-03T04:12:57.659Z DEBUG pub_sub::behaviour] [PubSubServiceBehaviour 1] channel 4086742429 add source 1
[2023-11-03T04:12:57.659Z INFO  pub_sub::behaviour] [PubSubServiceBehaviour 1] on channel 4086742429 added source 1
[2023-11-03T04:12:57.659Z DEBUG pub_sub::relay] [PubsubRelay] channel 4086742429 added source  1 => auto sub for local subs [0]
[2023-11-03T04:12:57.659Z INFO  pub_sub::relay::logic] [PubsubRelayLogic 1] local sub 4086742429/1 event from 0 pushed to list, new relay
[2023-11-03T04:12:57.665Z INFO  pub_sub::relay::logic] [PubsubRelayLogic 1] local sub 4086742429/1 event from 2 pushed to list
[2023-11-03T04:12:57.666Z INFO  pub_sub::behaviour] [PubSubServiceBehaviour 1] on added local channel 2861764290 => set Hashmap field
[2023-11-03T04:12:57.666Z DEBUG pub_sub::behaviour] [PubSubServiceBehaviour 1] channel 2861764290 add source 1
[2023-11-03T04:12:57.666Z INFO  pub_sub::behaviour] [PubSubServiceBehaviour 1] on channel 2861764290 added source 1
[2023-11-03T04:12:57.666Z DEBUG pub_sub::relay] [PubsubRelay] channel 2861764290 added source  1 => auto sub for local subs [1]
[2023-11-03T04:12:57.666Z INFO  pub_sub::relay::logic] [PubsubRelayLogic 1] local sub 2861764290/1 event from 1 pushed to list, new relay
[2023-11-03T04:12:57.668Z INFO  pub_sub::relay::logic] [PubsubRelayLogic 1] local sub 2861764290/1 event from 3 pushed to list
[2023-11-03T04:12:57.687Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 2861764290 from 1 to local 1 consumer not found
[2023-11-03T04:12:57.696Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 4086742429 from 1 to local 0 consumer not found
[2023-11-03T04:12:57.702Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 2861764290 from 1 to local 1 consumer not found
[2023-11-03T04:12:57.740Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 2861764290 from 1 to local 1 consumer not found
[2023-11-03T04:12:57.741Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 4086742429 from 1 to local 0 consumer not found
[2023-11-03T04:12:57.763Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 2861764290 from 1 to local 1 consumer not found
[2023-11-03T04:12:57.764Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 4086742429 from 1 to local 0 consumer not found
[2023-11-03T04:12:57.769Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 2861764290 from 1 to local 1 consumer not found
[2023-11-03T04:12:57.795Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 2861764290 from 1 to local 1 consumer not found
[2023-11-03T04:12:57.795Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 4086742429 from 1 to local 0 consumer not found
[2023-11-03T04:12:57.813Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 2861764290 from 1 to local 1 consumer not found
[2023-11-03T04:12:57.829Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 4086742429 from 1 to local 0 consumer not found
[2023-11-03T04:12:57.835Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 2861764290 from 1 to local 1 consumer not found
[2023-11-03T04:12:57.858Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 2861764290 from 1 to local 1 consumer not found
[2023-11-03T04:12:57.862Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 4086742429 from 1 to local 0 consumer not found
[2023-11-03T04:12:57.863Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 2861764290 from 1 to local 1 consumer not found
[2023-11-03T04:12:57.896Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 4086742429 from 1 to local 0 consumer not found
[2023-11-03T04:12:57.900Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 2861764290 from 1 to local 1 consumer not found
[2023-11-03T04:12:57.903Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 2861764290 from 1 to local 1 consumer not found
[2023-11-03T04:12:57.930Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 4086742429 from 1 to local 0 consumer not found
[2023-11-03T04:12:57.941Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 2861764290 from 1 to local 1 consumer not found
[2023-11-03T04:12:57.964Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 2861764290 from 1 to local 1 consumer not found
[2023-11-03T04:12:57.964Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 4086742429 from 1 to local 0 consumer not found
[2023-11-03T04:12:57.990Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 2861764290 from 1 to local 1 consumer not found
[2023-11-03T04:12:57.990Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 2861764290 from 1 to local 1 consumer not found
[2023-11-03T04:12:57.996Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 4086742429 from 1 to local 0 consumer not found
[2023-11-03T04:12:58.004Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 2861764290 from 1 to local 1 consumer not found
[2023-11-03T04:12:58.026Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 2861764290 from 1 to local 1 consumer not found
[2023-11-03T04:12:58.029Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 4086742429 from 1 to local 0 consumer not found
[2023-11-03T04:12:58.051Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 2861764290 from 1 to local 1 consumer not found
[2023-11-03T04:12:58.063Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 4086742429 from 1 to local 0 consumer not found
[2023-11-03T04:12:58.064Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 2861764290 from 1 to local 1 consumer not found
[2023-11-03T04:12:58.096Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 2861764290 from 1 to local 1 consumer not found
[2023-11-03T04:12:58.097Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 4086742429 from 1 to local 0 consumer not found
[2023-11-03T04:12:58.104Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 2861764290 from 1 to local 1 consumer not found
[2023-11-03T04:12:58.132Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 4086742429 from 1 to local 0 consumer not found
[2023-11-03T04:12:58.133Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 2861764290 from 1 to local 1 consumer not found
[2023-11-03T04:12:58.155Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 2861764290 from 1 to local 1 consumer not found
[2023-11-03T04:12:58.162Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 4086742429 from 1 to local 0 consumer not found
[2023-11-03T04:12:58.164Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 2861764290 from 1 to local 1 consumer not found
[2023-11-03T04:12:58.195Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 4086742429 from 1 to local 0 consumer not found
[2023-11-03T04:12:58.199Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 2861764290 from 1 to local 1 consumer not found
[2023-11-03T04:12:58.217Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 2861764290 from 1 to local 1 consumer not found
[2023-11-03T04:12:58.229Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 4086742429 from 1 to local 0 consumer not found
[2023-11-03T04:12:58.239Z WARN  pub_sub::relay::local] [LocalRelay] relay channel 2861764290 from 1 to local 1 consumer not found
[2023-11-03T04:12:58.239Z INFO  pub_sub::behaviour] [PubSubServiceBehaviour 1] on removed local channel 2861764290 => del Hashmap field
[2023-11-03T04:12:58.239Z INFO  pub_sub::behaviour] [PubSubServiceBehaviour 1] on removed local channel 4086742429 => del Hashmap field
[2023-11-03T04:12:58.239Z DEBUG pub_sub::behaviour] [PubSubServiceBehaviour 1] channel 2861764290 remove source 1
[2023-11-03T04:12:58.239Z DEBUG pub_sub::behaviour] [PubSubServiceBehaviour 1] channel 4086742429 remove source 1
[2023-11-03T04:12:58.239Z INFO  pub_sub::behaviour] [PubSubServiceBehaviour 1] on channel 2861764290 removed source 1
[2023-11-03T04:12:58.239Z DEBUG pub_sub::relay] [PubsubRelay] channel 2861764290 removed source 1 => auto unsub for local subs [1, 3]
[2023-11-03T04:12:58.239Z INFO  pub_sub::relay::logic] [PubsubRelayLogic 1] local unsub 2861764290/1 event from 1 removed from list
[2023-11-03T04:12:58.239Z INFO  pub_sub::relay::logic] [PubsubRelayLogic 1] local unsub 2861764290/1 event from 3 removed from list
[2023-11-03T04:12:58.239Z INFO  pub_sub::relay::logic] [PubsubRelayLogic 1] local unsub 2861764290/1 event from 3 list empty in source node => removed
[2023-11-03T04:12:58.239Z INFO  pub_sub::behaviour] [PubSubServiceBehaviour 1] on channel 4086742429 removed source 1
[2023-11-03T04:12:58.239Z DEBUG pub_sub::relay] [PubsubRelay] channel 4086742429 removed source 1 => auto unsub for local subs [0, 2]
[2023-11-03T04:12:58.239Z INFO  pub_sub::relay::logic] [PubsubRelayLogic 1] local unsub 4086742429/1 event from 0 removed from list
[2023-11-03T04:12:58.239Z INFO  pub_sub::relay::logic] [PubsubRelayLogic 1] local unsub 4086742429/1 event from 2 removed from list
[2023-11-03T04:12:58.239Z INFO  pub_sub::relay::logic] [PubsubRelayLogic 1] local unsub 4086742429/1 event from 2 list empty in source node => removed

Roadmap

Monitor

  • Dashboard #58
  • Prometheus endpoint #58
  • System metrics #58
  • Node metrics #58
  • Routing table metrics #58

Protocol:

  • scoped routing/service #47
  • public/private node-id #48
  • nat-traversal
  • encryption
  • service broadcast #46

Services:

  • Key-value scoped value #47
  • Key-value replication #45
  • Manual discovery with node-tags (scoped or global) #49
  • DHT discovery
  • Pubsub multi layers #50
  • Rpc Service #87

Latency, Throughput, and Scalability question

Hello,

I am researching various P2P projects for a low-latency, high-throughput, and hugely scaling project that might be considered to be used as the core to build out a unique type of P2P network in which data (i.e. files, streams, database, etc.) will be spread across the network and users will interact in various groups on the network which might be similar to the "rooms" in your atm0s-sdn project.

Originally the idea was to use Wireguard VPN tunnels along with STUN/TURN relays for computers behind NAT firewalls as to a possibly VPN type typical solution, but came across your project and it made me start to think outside-the-box to see is perhaps something new might be possible.

Recently, I also looked at your "Decentralized Ultra-Low Latency Streaming Server" (https://github.com/8xFF/atm0s-media-server) and am very impressed with what I saw there as well.

Any thoughts or suggestions would be greatly appreciated.

I would also be very interested to see any network benchmarks that you might have as well.

Thanks in advance and Happy New Year.

using ebpf for speeding-up

We can use ebpf for

But this method only work with UDP transport without p2p encryption.

TODO:

  • Implement OffloadRouting Mechanism
  • Implement Rewrite for routing rule
  • Implement bpf_clone_redirect for PubSub

chore: add missing info to publish to crates.io

Release-plz return bellow error

the remote server responded with an error: missing or empty metadata fields: description, license. Please see https://doc.rust-lang.org/cargo/reference/manifest.html for more information on configuring these fields

Improve README.md

This issue keeps track of the improvements needed for README.md

  • Improve on the wording and phrasing
  • Logo (optional?)
  • Architecture overview (ARCHITECTURE.md)
  • Change logs (CHANGELOGS.md)
  • Contribution guide (CONTRIBUTING.md)
  • Comparison with other solutions
  • Examples
  • Benchmark (optional?)'
  • Demos

Node status dashboard

For monitoring performance and current status of node, we need a simple dashboard like other node

  • simple dashboard like
    image

  • prometheus exporter

Metric:

  • node info (cpu, memory, network)
  • live pubsubs
  • live keys
  • live nodes
  • live connections

Log:

  • list connections
  • list nodes
  • list pubsubs, keys

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

cargo
Cargo.toml
  • bincode 1.3.3
  • serde 1.0
  • thiserror 1.0
  • log 0.4
  • rand 0.8
  • parking_lot 0.12
  • env_logger 0.11
  • clap 4.4
  • mockall 0.12.1
  • num_enum 0.7.2
  • convert-enum 0.1.0
  • sans-io-runtime 0.1.0
bin/Cargo.toml
  • signal-hook 0.3.17
  • tokio 1
  • poem 2.0
  • rust-embed 8.2
  • futures-util 0.3
  • tracing-subscriber 0.3
  • serde_json 1.0
fuzz/Cargo.toml
  • libfuzzer-sys 0.4
packages/core/identity/Cargo.toml
  • multiaddr 0.18.1
  • rand 0.8
packages/core/router/Cargo.toml
  • criterion 0.5.1
  • rand 0.8.5
packages/core/utils/Cargo.toml
packages/network/Cargo.toml
  • bytes 1.5
  • bincode 1.3
  • sha1 0.10
  • num 0.4
  • sha2 0.10
  • x25519-dalek 2.0
  • aes-gcm 0.10
  • derivative 2.2
packages/runner/Cargo.toml
  • signal-hook 0.3.17
dockerfile
Dockerfile
  • ubuntu 22.04
  • ubuntu 22.04
github-actions
.github/workflows/release-plz.yml
  • actions/checkout v4
  • MarcoIeni/release-plz-action v0.5
.github/workflows/release.yml
  • actions/checkout v4
  • actions-rs/toolchain v1
  • actions-rs/cargo v1
  • actions/upload-artifact v4
  • svenstaro/upload-release-action v2
  • actions/download-artifact v4
  • docker/setup-qemu-action v3
  • docker/setup-buildx-action v3
  • actions/checkout v4
  • actions/download-artifact v4
  • docker/login-action v3
  • docker/metadata-action v5
  • docker/build-push-action v5
.github/workflows/rust-clippy.yml
  • actions/checkout v4
  • actions/cache v3
  • arduino/setup-protoc v3
  • github/codeql-action v3
.github/workflows/rust-fmt.yml
  • actions/checkout v4
  • actions/cache v3
.github/workflows/rust.yml
  • actions/checkout v4
  • actions/cache v3
  • arduino/setup-protoc v3
  • codecov/codecov-action v4
  • actions/checkout v4
  • EmbarkStudios/cargo-deny-action v1

  • Check this box to trigger a request for Renovate to run again on this repository

HTTP Based Discovery as an alternative to MultiAddress

In the current state, in order to initialize a cluster, a seed in a form of a multiaddr will be required. This seed can only be decided when there're at least 1 node already running (unpredictable port and ip addresses). Due to this limitation, we cannot seamlessly spin up a cluster without having some overhead logic for acquiring the initial seed.
So instead of using multiaddr explicitly, we can opt for an HTTP Based Discovery using API Endpoints somehow.

Scoped Service

With idea about some node that running a service but it only serve for a specific scope: some nodes inside a netmask (same with ip-v4 netmask)

Bug: pubsub channel timeout

Describe the bug
Suddenly pubsub change stop and have bellow error:

2024-05-15T15:40:33.115948Z  WARN atm0s_sdn_network::features::pubsub::controller::source_hint: [SourceHint] Notify remove source(2) to local [Worker(0, ClusterRoomHash(5481250879105948498))] actors because timeout
2024-05-15T15:40:33.115982Z  INFO atm0s_sdn_network::features::pubsub::controller: [PubSubFeatureController] UnsubSource(source) for ChannelId(17142674879314892472) from Worker(0, ClusterRoomHash(5481250879105948498))

To Reproduce
Unknown

Expected behavior
Should not happend

add more docs

Write more info about project, technology. Design architecture

Extend to Public/Private node-id

In some applications we need some private local nodes, which only contact with some relay nodes. Example media-servers, IoT devices. By that way we can limit the number of connections in each nodes.

Transporter should limit incoming packet size

Describe the bug
The current TcpTransporter is allowing unlimited packet size when handshake. This can cause a spike in memory allocation when a malicious actor send a huge packet, if the size if big enough can resulted in a DDOS attack.

Expected behavior
The incoming packet should be limited in size.

Screenshots
image

Replicate Key-Value storage

Each key in key-value storage is only saved in single node in current version, it should be replicate at least 2 nodes. It is better to configurable how many node it is replicated.

May be it is affected to bellow parts:

  • Set destinations and how to determine that set is success (now version is only single flag acked)
  • Sub destinations and how to determine that sub is success (now version is only single flag acked)
  • Sync destinations
  • Get destinations

May be the simplest way (any ack mean success) is the best in decentralized environments; because if it not success that time, it will be synced after that.

Bug: transport secure mode failed to receive message

Describe the bug
When we testing latency with VPN service, sometime the ping timeout, after tracing I found that the secure context of udp transport (may be both udp and tcp) cannot decrypt data. I dont know why but have some potential problems:

  • Lost packet
  • Unorder

To Reproduce
Steps to reproduce the behavior:

  1. Start Node A with vpn
  2. Start Node B with vpn (bad network conditions, maybe can simulate)
  3. From node A ping node B (in very rarely condition, ping timeout)

Expected behavior
All of ping should be successl

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.