Giter VIP home page Giter VIP logo

libmcrx's Introduction

Intro

libmcrx is a low-level library for receiving multicast UDP traffic.

Build Clang Static Analysis

Motivation

The code for receiving multicast traffic is different on different platforms, and has some quirks and complexities. This library is intended to ease adoption of multicast-based protocols by simplifying the programming task of receiving multicast packets at the client side. Its API uses C linkage in an attempt to maximize the portability for implementing simple wrappers in other languages.

The library also is intended to serve as an extension point to integrate with some standards and standard-tracked work in progress in the IETF, ideally including:

  • AMBI: an asymmetric cryptographic authentication scheme for multicast traffic
    • this provides loss statistics as well as authenticated payloads
  • CBACC: bandwith limitation enforcement.
  • AMT: unicast tunneling
    • with DRIAD, a tunnel discovery mechanism that ensures use of native multicast is favored where possible

Building

Linux

Prerequisites

For normal linux builds of the library, this project uses autoconf and automake, following the libabc template.

APT-based (Debian, Ubuntu)

apt-get install \
  autoconf \
  libtool-bin \
  make \
  build-essential

MacOS with brew

Using brew:

brew install \
  autoconf \
  automake \
  make \
  libtool

TBD: more platforms

Build

./autogen.sh
./configure
make

As usual with autotools, ./configure --help provides a bunch of options, and more in-depth explanations give more useful details.

For example, with something like --prefix=${HOME}/local-install a make install will not need sudo, and will put the library and header files under ${HOME}/local-install.

Test

Note that to test this library, you'll need reachability to an active sender of multicast traffic. See the how-to for some approaches.

NB: the current tests use a hardcoded (S,G), and probably should be changed to use a config file instead. As-is you'll basically need to be using the multicast-ingest-platform or to be in a multicast-capable network that performs ingest using DRIAD for it to pass, assuming the sender it uses is running.

make check

Install

Autoconf by default should build a configure and Makefiles that will put the headers and libraries into the default location for user libraries for the system.

sudo make install

Using the library

This library is structured as a few types of objects in a hierarchy, wrapping as much socket-receiving complexity as we could arrange.

The relevant objects are:

  • ctx: a context. All the function calls are associated with exactly one context (or with an object below that's associated with exactly one context). Multiple contexts may exist, and objects associated with different contexts do not directly interact (they can be processed by separate threads without any synchronization between contexts, for example).
  • sub: a subscription. Each subscription is associated immutably with an (S,G) or (*,G) and a UDP port number. A callback function is set for the subscription and provides received packets.
  • pkt: a packet. Each packet provides the payload of a UDP packet received on the wire and associated with a particular subscription.

Each of these objects is created with a "_new" function, and is destroyed with a "_unref" function. (An internal refcount may be increased with a "_ref" function, which will keep the object alive through one extra "_unref".)

Each object also can hold an arbitrary "user data" pointer, set and retrieved with a set_userdata and get_userdata function. That pointer is opaque to the library, and provided as a convenience for the calling system.

Basic usage looks like this:

#include <mcrx/libmcrx.h>

static int receive_cb(struct mcrx_packet* packet);

int receive_thread() {
  struct mcrx_ctx* ctx = NULL;
  mcrx_ctx_new(&ctx);

  struct mcrx_subscription_config conf = MCRX_SUBSCRIPTION_CONFIG_INIT;
  mcrx_subscription_config_pton(&conf, "23.212.185.4", "232.1.1.1");
  conf.port = 5001;

  struct mcrx_subscription *sub;
  mcrx_subscription_new(ctx, &conf, &sub);
  mcrx_subscription_set_receive_cb(sub, receive_cb);

  mcrx_subscription_join(sub);

  while (1) {
    mcrx_ctx_receive_packets(ctx);
  }
}

static int receive_cb(struct mcrx_packet* packet) {
  // do something with packet
  // operations like creating, joining, and leaving subscriptions are
  // safe here.
  mcrx_packet_unref(packet);
  return MCRX_RECEIVE_CONTINUE;
}

For a more detailed example including appropriate error handling, please see mcrx-check.

Thread Safety

There is no thread safety handling inside the library. It's the caller's responsibility to ensure that no calls to any functions using the same ctx or the objects generated from the same ctx (including packets and subscriptions) have function calls that overlap in time between different threads.

Alternative Event Handlers

For an example integrating with an external event handler instead of using the blocking mcrx_ctx_receive_packets call, see the libmcrx integration with python-asyncio-taps.

That project uses python's asyncio as the event handling library, and exports sockets to be added to the list of sockets to monitor for read readiness via mcrx_ctx_set_receive_socket_handlers, rather than using the blocking mcrx_ctx_receive_packets call.

In that scenario, the calling system is responsible for making a timely call to the do_receive function for all the sockets that have been given to the calling system with the add_socket_cb callback and that have not yet been removed with the remove_socket_cb callback.

libmcrx's People

Contributors

grumpyoldtroll avatar ghenry avatar bqiuakamai avatar maxf12 avatar squarooticus avatar

Stargazers

Agostino avatar dikshie avatar  avatar Gavin Lefebvre avatar  avatar

Watchers

 avatar  avatar James Cloos avatar  avatar  avatar  avatar

libmcrx's Issues

GitHub Action

Add an action to build and clean the project on push etc.

Removing unused headers and functions

Some of these includes can be deleted, but I'm presuming the unused functions are for users of the library?

Screenshot from 2022-02-03 12-20-19

We could clear these warnings by using them in a test cases if possible.

family value never read

src/native.c:358:7: warning: Value stored to 'family' is never read [deadcode.DeadStores]
      family = AF_INET;
      ^        ~~~~~~~

Update configure.ac

  • Run autoupdate
  • Run autoscan to find all missing checks.

Pull Request coming.

Make coverity scans conditional on changes

Right now coverity scans are scheduled daily unconditional, so it's often re-scanning an unchanged repo:

- cron: '0 11 * * *' # Daily at 11:00 UTC

I think re-scanning is a good idea if the coverity version changes or the repo changes, but ideally not otherwise.

Once it's a tighter conditional, it would also be good to scan PRs before integrating them if possible, and maybe make the check more frequent. The changes to this repo are infrequent enough that it's probably better to leverage the 7/day max limit differently than a hard once per day.

App receives packets from the wrong source address on a group+port collision.

I noticed this while running MNAT on the receiver. Packets arriving on the physical interface with a different source address than the subscribed (S,G) were received instead of filtered in the library. (The OS can't filter it because the listening UDP socket is only bound to the group and the port; even though the sockopt is an SSM join, that's not a filter for the receiving port.)

Add "make check" runs in repo/pr automation

I've been running "make check" before commits, but it requires some local config before running. On linux, assuming libmcrx is in ~/src, that looks like:

docker container run -d --rm --name amtgw-5 --privileged grumpyoldtroll/amtgw $(python3 ~/src/libmcrx/driad.py 23.212.185.5)
sudo ip route add 23.212.185.5/32 dev docker0

Then, as long as my sender of traffic from 23.212.185.5->232.1.1.1:5001 is alive and properly registered in the DNS and reachable, the tests pass. (Else they hang forever, which maybe also should be fixed to fail after some time.)

Ideally, this would also be coupled with adding valgrind to the tests and failing on violations. Some links on how to do so are here:

Other nice-to-haves would be gcov or similar, with some kind of coverage report shown on pages (also mentioned in some of the above but probably more research needed.)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.