Giter VIP home page Giter VIP logo

mediagateway's Introduction

Media Gateway

The media gateway provides a functionality to forward messages from one ZeroMQ instance to another. The media gateway consists of two applications - a server and client. The client reads messages from the source ZeroMQ instance and sends them to the server via HTTP/HTTPS. The server writes received messages to the target ZeroMQ instance.

Following optional features are supported:

  • basic authentication
  • HTTPS (including a self-signed PEM encoded certificate)
  • client certificate authentication (for the server X509_LOOKUP_hash_dir method is used to load certificates and CRLs)
  • FPS statistics logging (by frame or timestamp period)

To read from and to write to ZeroMQ savant_core crate is used.

Both server and client applications are configured via JSON files. in_stream in the client configuration corresponds to ReaderConfig and out_stream in the server configuration corresponds to WriterConfig. Examples of configuration files can be found in samples directory.

Both server and client have a health endpoint.

 GET /health HTTP/1.1
 Host: <host>

If the server/client is healthy an HTTP response with 200 OK status code and the body as below will be returned.

{
 "status": "healthy"
}

Docker

Both server and client can be run as Docker containers.

Server

To run the server with the default configuration and to mount /tmp directory and publish the port from the default configuration

docker run \
 -v /tmp:/tmp \
 -p 8080:8080 \
 ghcr.io/insight-platform/media-gateway-server:latest

To run the server with another configuration (/home/user/server_config.json)

docker run \
 -v /home/user/server_config.json:/opt/etc/custom_config.json \
 -p HOST_PORT:CONFIG_PORT \
 ghcr.io/insight-platform/media-gateway-server:latest \
 /opt/etc/custom_config.json

Client

To run the client with the default configuration, to mount /tmp directory and publish the port from the default configuration

docker run \
 -v /tmp:/tmp \
 -p 8081:8081 \
  -e "GATEWAY_URL=<GATEWAY_URL>" \
 ghcr.io/insight-platform/media-gateway-client:latest

where <GATEWAY_URL> is the server URL, e.g. http://192.168.0.100:8080

To run the server with another configuration (/home/user/client_config.json)

docker run \
 -v /home/user/client_config.json:/opt/etc/custom_config.json \
 -p <HOST_PORT>:<CONFIG_PORT> \
 ghcr.io/insight-platform/media-gateway-client:latest \
 /opt/etc/custom_config.json

where <CONFIG_PORT> is the port specified in the configuration file and <HOST_PORT> is the port on the host machine.

mediagateway's People

Contributors

ksenia-vazhdaeva avatar bwsw avatar

Watchers

 avatar  avatar

mediagateway's Issues

Replace basic authentication with ACCESS_KEY, SECRET_KEY authentication and Etcd

We must manage users based on a more traditional approach in cloud environments (AWS): we must remove HTTP Basic Auth and replace it with header-based ACCESS_KEY and SECRET_KEY authentication.

The server must use Etcd to access the authentication information. We already have the code implementing Etcd watched behavior: https://github.com/insight-platform/etcd_dynamic_state

Usage sample: https://github.com/insight-platform/savant-rs/blob/main/savant_core/src/eval_resolvers.rs#L256

This Etcd client implementation uses Etcd watch to get updates on all changes under the prefix and cache them locally. Thus, it can be used to implement such authentication.

Structure:

prefix
  - AC_1
  - AC_2

Value contains the following contents:

{
  "secret_key": "goes here", // salty, hash, not a plain value
  "allowed_source_ids": [..., ...], // globs (https://docs.rs/glob/latest/glob/)
  "allowed_routing_labels": ["ab", "cde", "label-1"]
}

Authentication must be implemented in a smart way to avoid running encryption functions when the same AK/SK is passed. For that, we can use the LRU cache (https://docs.rs/lru/latest/lru/) to keep the recalculated values in the memory.

The configuration file must configure the cache's size. The system must also support tracking LRU evicted keys, and a warning must be produced when more than X keys/sec are evicted.

Cleanup statistics cofiguration

It is unlikely to use both statistics strategies (by a frame period, by a timestamp period) at the same time. Moreover incorrectly configured history size might confuse the user as not all statistics will be reported to logs. E.g. with real FPS 3000, frame period 1000, timestamp period 1s, history size 4 statistics by timestamp will not be in logs at all.

history_size does not make sense for the user and might be removed if only one strategy is allowed.

The "client-certificate-authentication" section in docs is not self-explanatory

Needs rework. It is absolutely not clear how the certificate lifecycle works. Why does CA require CRL? What about sequential numbers?

The manual MUST provide a self-containing guide on how to configure the client and server end-to-end without looking to external docs. Structured, with sections and steps.

The docs must be developed in the form of a tutorial.

  1. how does it work;
  2. structure of files and configurations;
  3. how create a client certificate;
  4. how to add a client certificate;
  5. how to revoke a client certificate.

Add retry delay exponential wait strategy

When the system fails to send a message, the delay must increase twice until the upper bond is reached, e.g.,

1ms
2ms
4ms
16ms
increase while < configured cap

to avoid resource waste and log hell. When it fails, the system must log when the next retry will happen. After the successful delivery, the delay is reset.

Update documentation

Describe AAA models:

  • TLS+basic (encryption)
  • TLS+X509+CRL (encryption, authentication, revocation list)
  • TLS+X509+CRL+basic (encryption, authentication, revocation list, AK/SK authentication)

High CPU usage by media gateway client

CPU load by media gateway client is high.

Use cases:

  • e2e_usage_video_loop_ao_rtsp 170%
  • multisource benchmark (SYNC_OUTPUT=true, NUMBER_OF_STREAMS=128) 190%

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.