usnavalresearchlaboratory / norm Goto Github PK

View Code? Open in Web Editor NEW

93.0 16.0 33.0 13.27 MB

NACK-Oriented Reliable Multicast (NORM) implementation & tools (RFCs 5740, 5401)

Home Page: https://www.nrl.navy.mil/Our-Work/Areas-of-Research/Information-Technology/NCS/NORM/

License: Other

C++ 91.77% C 1.38% Tcl 0.72% Makefile 0.08% Python 3.02% Java 1.33% HTML 1.17% CMake 0.43% Dockerfile 0.02% Shell 0.06%

transport-api reliable-data-transfer reliable-networking reliable-protocol multicast reliable-udp-library

norm's Introduction

norm

                    NORM SOURCE CODE RELEASE

/*********************************************************************
 *
 * AUTHORIZATION TO USE AND DISTRIBUTE
 * 
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that: 
 *
 * (1) source code distributions retain this paragraph in its entirety, 
 *  
 * (2) distributions including binary code include this paragraph in
 *     its entirety in the documentation or other materials provided 
 *     with the distribution.
 * 
 *      "This product includes software written and developed 
 *       by Code 5520 of the Naval Research Laboratory (NRL)." 
 *         
 *  The name of NRL, the name(s) of NRL  employee(s), or any entity
 *  of the United States Government may not be used to endorse or
 *  promote  products derived from this software, nor does the 
 *  inclusion of the NRL written and developed software  directly or
 *  indirectly suggest NRL or United States  Government endorsement
 *  of this product.
 * 
 * THIS SOFTWARE IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR
 * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED
 * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.
 *
 ********************************************************************/

This is the NRL NORM source code repository.

The Github "Discussions" feature is available here for comment and question in addition to the regular code issue reporting:

https://github.com/USNavalResearchLaboratory/norm/discussions

SOURCE CODE

The "norm" source Git repository includes "protolib" as a git submodule. You can add the "--recurse-submodules" to your git clone command to automate inclusion:

git clone --recurse-submodules https://github.com/USNavalResearchLaboratory/norm.git

The following items can be built from this source code release:

libnorm.a - static NORM library for Unix platforms, or

Norm.lib - static NORM protocol library for Win32 platforms

(These libraries can be used by applications using the NORM API. The file "normApi.h" provides function prototypes for NORM API calls and the included "Norm Developer's Guide" provides a reference manual for NORM application development using this API. Additional tutorial material and API usage examples will be provided as additional documentation in the future)

norm (or norm.exe (WIN32) - command-line "demo" application built from "common/normApp.cpp"

(The included "Norm User's Guide" provides a rough overview of how to use this demo app. This demo application is useful for file transfers (and streaming on Unix))

normTest (or normTest.exe (WIN32)) - very simple test application

(The "normTest" application (see "common/normTest.cpp") is really just a simple playground for preliminary testing of the NORM API and does not do anything useful. But it does provide a very simple example of NORM API usage. More sophisticated (and better-documented) examples of NORM API usage will be provided in the future.

There is also an "examples" directory in the source code tree that contains some simplified examples of NORM API usage. The example set will be expanded as time progresses.

NRL also has started some apps built around NORM including:

A) A reliable multicast "chat" application (users can share files and images, too) with a GUI. Like IRC with no server needed.

B) An application for reliably "tunneling" real-time UDP packet streams (like RTP video or audio) using NORM's streaming capability.

NACK-Oriented Proxy (NORP)

NORM Proxy code is also included in the 'norp' directory. More info is available in the norp/doc directory, as well as the NORP website: https://www.nrl.navy.mil/itd/ncs/products/norp

Docker

A Dockerfile is provided to facilitate containerized NORM environment testing. Notable build arguments passed to docker build will influence how the image is created.

platformName represents the base image name (defaults to ubuntu)
platformVersion represents the tagged version for the base image (defaults to latest)
version is a 1:1 mapping to this repo's tag (defaults to v1.5.9)
configure is piped to ./waf configure (defaults to "")
build is piped to ./waf (defaults to --targets=*)

Example usage:

CentOS7 build

docker build --build-arg platformName=centos \
  --build-arg platformVersion=7 \
  --build-arg="--build-java" \
  -t norm:centos-1.5.9 .

Running the data send and recv examples:

# Sender in one terminal
docker run -it --rm --name norm-send <norm:image> normDataSend
# Receiver in another terminal
docker run -it --rm --name norm-recv <norm:image> normDataRecv

OTHER FILES:

NormDeveloperGuide.pdf - PDF version of NORM Developer's Guide (nice for printing)

NormDeveloperGuide.html - HTML version of NORM Developer's Guide with Hyperlinked content (nice for browsing)

NormUserGuide.pdf - Guide to "norm" demo app usage

VERSION.TXT - NORM version history notes

README.md - this file

docker - directory containing docker container support

NOTES:

The NORM code depends upon the current "Protolib" release:

https://github.com/USNavalResearchLaboratory/protolib

It has been addded as a git submodule to the NORM git repository. So, to to build you will need to do the following steps to download the protolib code:

git clone --recurse-submodules https://github.com/USNavalResearchLaboratory/norm.git

Alternatively after a basic "git clone" you can do the folowing to pull in the protolib source:

cd norm git submodule update --init

To keep the 'protolib' submodule up to date, you will need to do the following:

cd norm/protolib git checkout master

This will enable you to issue 'git pull', etc to treat the 'protolib' sub-directory as its own (sub-) repository.

norm's People

Contributors

Stargazers

Watchers

norm's Issues

Question：how to get the status of each sender session?

the real send rate, function getTxRate return is always too high or too low;
how much bytes sended of an object(especially for a large file);
the numbers of parity symbol segments (packets)/source symbol segments/resend segments in an interval, the interval values can be settled by API. Each interval emits an event? used to calculate packet loss ratio?

FEC Object Transmission Information format for FEC Encoding ID 5 appears to deviate from RFC 5510

Per RFC 5510, section 5.2, the format of the EXT_FTI header extension for FEC Encoding ID 5 is:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   HET = 64    |    HEL = 3    |                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
   |                      Transfer Length (L)                      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   Encoding Symbol Length (E)  | MaxBlkLen (B) |     max_n     |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

where max_n is defined as:

the maximum number of encoding symbols generated for any source block

According to section 6.2, max_n = ceil (B / CR) where CR is the code rate.

However, member function SetFecNumParity() of class NormFtiExtension5 appears to write the number of generated repair symbols (n_r in RFC 5510 notation) into this field of the FTI extension header instead.

Building release v1.5.8 with protolib v3.0b1 results in build errors

I'm trying to build libnorm version 1.5.8 that previously worked fine when I obtained the source tarball from https://downloads.pf.itd.nrl.navy.mil/norm/.

When I'm trying to build libnorm with the latest (only) protolib release I get build errors.

When I'm building libnorm with protolib master everything builds fine.

What version of protolib should I use? I assume protolib master but it is not made fully clear from the documentation.

Thanks (build log included below)

user@node:/build/norm-1.5.8/makefiles$ make -j 4 -f Makefile.linux libnorm.so
g++ -c -g -DPROTO_DEBUG -DUNIX -D_FILE_OFFSET_BITS=64 -O -Wall -Wcast-align -pedantic -fPIC -Wno-variadic-macros -DLINUX -DECN_SUPPORT  -DHAVE_IPV6 -DHAVE_GETLOGIN -D_FILE_OFFSET_BITS=64 -DHAVE_LOCKF -DHAVE_OLD_SIGNALHANDLER -DHAVE_DIRFD -DHAVE_ASSERT -DNO_SCM_RIGHTS -DHAVE_SCHED -DUNIX -DUSE_SELECT -DUSE_TIMERFD -DUSE_EVENTFD -DHAVE_PSELECT -Wall -I/usr/X11R6/include  -I../src/unix -I../include -I../protolib/include -o ../src/common/normMessage.o ../src/common/normMessage.cpp
g++ -c -g -DPROTO_DEBUG -DUNIX -D_FILE_OFFSET_BITS=64 -O -Wall -Wcast-align -pedantic -fPIC -Wno-variadic-macros -DLINUX -DECN_SUPPORT  -DHAVE_IPV6 -DHAVE_GETLOGIN -D_FILE_OFFSET_BITS=64 -DHAVE_LOCKF -DHAVE_OLD_SIGNALHANDLER -DHAVE_DIRFD -DHAVE_ASSERT -DNO_SCM_RIGHTS -DHAVE_SCHED -DUNIX -DUSE_SELECT -DUSE_TIMERFD -DUSE_EVENTFD -DHAVE_PSELECT -Wall -I/usr/X11R6/include  -I../src/unix -I../include -I../protolib/include -o ../src/common/normSession.o ../src/common/normSession.cpp
g++ -c -g -DPROTO_DEBUG -DUNIX -D_FILE_OFFSET_BITS=64 -O -Wall -Wcast-align -pedantic -fPIC -Wno-variadic-macros -DLINUX -DECN_SUPPORT  -DHAVE_IPV6 -DHAVE_GETLOGIN -D_FILE_OFFSET_BITS=64 -DHAVE_LOCKF -DHAVE_OLD_SIGNALHANDLER -DHAVE_DIRFD -DHAVE_ASSERT -DNO_SCM_RIGHTS -DHAVE_SCHED -DUNIX -DUSE_SELECT -DUSE_TIMERFD -DUSE_EVENTFD -DHAVE_PSELECT -Wall -I/usr/X11R6/include  -I../src/unix -I../include -I../protolib/include -o ../src/common/normNode.o ../src/common/normNode.cpp
g++ -c -g -DPROTO_DEBUG -DUNIX -D_FILE_OFFSET_BITS=64 -O -Wall -Wcast-align -pedantic -fPIC -Wno-variadic-macros -DLINUX -DECN_SUPPORT  -DHAVE_IPV6 -DHAVE_GETLOGIN -D_FILE_OFFSET_BITS=64 -DHAVE_LOCKF -DHAVE_OLD_SIGNALHANDLER -DHAVE_DIRFD -DHAVE_ASSERT -DNO_SCM_RIGHTS -DHAVE_SCHED -DUNIX -DUSE_SELECT -DUSE_TIMERFD -DUSE_EVENTFD -DHAVE_PSELECT -Wall -I/usr/X11R6/include  -I../src/unix -I../include -I../protolib/include -o ../src/common/normObject.o ../src/common/normObject.cpp
In file included from ../protolib/include/protoAddress.h:8,
                 from ../protolib/include/protoSocket.h:4,
                 from ../protolib/include/protokit.h:6,
                 from ../include/normMessage.h:5,
                 from ../src/common/normMessage.cpp:1:
../protolib/include/protoTree.h: In member function ‘ITEM_TYPE* ProtoSortedTreeTemplate<ITEM_TYPE>::Iterator::PeekPrevItem() const’:
../protolib/include/protoTree.h:664:93: error: no matching function for call to ‘ProtoSortedTreeTemplate<ITEM_TYPE>::Iterator::PeekPrevItem() const’
                     {return static_cast<ITEM_TYPE*>(ProtoSortedTree::Iterator::PeekPrevItem());}
                                                                                             ^
../protolib/include/protoTree.h:566:23: note: candidate: ‘ProtoSortedTree::Item* ProtoSortedTree::Iterator::PeekPrevItem()’ <near match>
                 Item* PeekPrevItem()
                       ^~~~~~~~~~~~
../protolib/include/protoTree.h:566:23: note:   passing ‘const ProtoSortedTreeTemplate<ITEM_TYPE>::Iterator*’ as ‘this’ argument discards qualifiers
../protolib/include/protoTree.h: In member function ‘ITEM_TYPE* ProtoSortedTreeTemplate<ITEM_TYPE>::Iterator::PeekNextItem() const’:
../protolib/include/protoTree.h:669:93: error: no matching function for call to ‘ProtoSortedTreeTemplate<ITEM_TYPE>::Iterator::PeekNextItem() const’
                     {return static_cast<ITEM_TYPE*>(ProtoSortedTree::Iterator::PeekNextItem());}
                                                                                             ^
../protolib/include/protoTree.h:564:23: note: candidate: ‘ProtoSortedTree::Item* ProtoSortedTree::Iterator::PeekNextItem()’ <near match>
                 Item* PeekNextItem()
                       ^~~~~~~~~~~~
../protolib/include/protoTree.h:564:23: note:   passing ‘const ProtoSortedTreeTemplate<ITEM_TYPE>::Iterator*’ as ‘this’ argument discards qualifiers
In file included from ../protolib/include/protoAddress.h:8,
                 from ../protolib/include/protoSocket.h:4,
                 from ../protolib/include/protokit.h:6,
                 from ../include/normMessage.h:5,
                 from ../include/normNode.h:4,
                 from ../src/common/normNode.cpp:1:
../protolib/include/protoTree.h: In member function ‘ITEM_TYPE* ProtoSortedTreeTemplate<ITEM_TYPE>::Iterator::PeekPrevItem() const’:
../protolib/include/protoTree.h:664:93: error: no matching function for call to ‘ProtoSortedTreeTemplate<ITEM_TYPE>::Iterator::PeekPrevItem() const’
                     {return static_cast<ITEM_TYPE*>(ProtoSortedTree::Iterator::PeekPrevItem());}
                                                                                             ^
../protolib/include/protoTree.h:566:23: note: candidate: ‘ProtoSortedTree::Item* ProtoSortedTree::Iterator::PeekPrevItem()’ <near match>
                 Item* PeekPrevItem()
                       ^~~~~~~~~~~~
In file included from ../protolib/include/protoAddress.h:8,
                 from ../protolib/include/protoSocket.h:4,
                 from ../protolib/include/protokit.h:6,
                 from ../include/normMessage.h:5,
                 from ../include/normSegment.h:4,
                 from ../include/normObject.h:4,
                 from ../src/common/normObject.cpp:1:
../protolib/include/protoTree.h: In member function ‘ITEM_TYPE* ProtoSortedTreeTemplate<ITEM_TYPE>::Iterator::PeekPrevItem() const’:
../protolib/include/protoTree.h:664:93: error: no matching function for call to ‘ProtoSortedTreeTemplate<ITEM_TYPE>::Iterator::PeekPrevItem() const’
                     {return static_cast<ITEM_TYPE*>(ProtoSortedTree::Iterator::PeekPrevItem());}
                                                                                             ^
../protolib/include/protoTree.h:566:23: note:   passing ‘const ProtoSortedTreeTemplate<ITEM_TYPE>::Iterator*’ as ‘this’ argument discards qualifiers
../protolib/include/protoTree.h: In member function ‘ITEM_TYPE* ProtoSortedTreeTemplate<ITEM_TYPE>::Iterator::PeekNextItem() const’:
../protolib/include/protoTree.h:669:93: error: no matching function for call to ‘ProtoSortedTreeTemplate<ITEM_TYPE>::Iterator::PeekNextItem() const’
                     {return static_cast<ITEM_TYPE*>(ProtoSortedTree::Iterator::PeekNextItem());}
                                                                                             ^
../protolib/include/protoTree.h:564:23: note: candidate: ‘ProtoSortedTree::Item* ProtoSortedTree::Iterator::PeekNextItem()’ <near match>
                 Item* PeekNextItem()
                       ^~~~~~~~~~~~
../protolib/include/protoTree.h:564:23: note:   passing ‘const ProtoSortedTreeTemplate<ITEM_TYPE>::Iterator*’ as ‘this’ argument discards qualifiers
../protolib/include/protoTree.h:566:23: note: candidate: ‘ProtoSortedTree::Item* ProtoSortedTree::Iterator::PeekPrevItem()’ <near match>
                 Item* PeekPrevItem()
                       ^~~~~~~~~~~~
In file included from ../protolib/include/protoAddress.h:8,
                 from ../protolib/include/protoSocket.h:4,
                 from ../protolib/include/protokit.h:6,
                 from ../include/normMessage.h:5,
                 from ../include/normSession.h:4,
                 from ../src/common/normSession.cpp:1:
../protolib/include/protoTree.h: In member function ‘ITEM_TYPE* ProtoSortedTreeTemplate<ITEM_TYPE>::Iterator::PeekPrevItem() const’:
../protolib/include/protoTree.h:664:93: error: no matching function for call to ‘ProtoSortedTreeTemplate<ITEM_TYPE>::Iterator::PeekPrevItem() const’
                     {return static_cast<ITEM_TYPE*>(ProtoSortedTree::Iterator::PeekPrevItem());}
                                                                                             ^
../protolib/include/protoTree.h:566:23: note: candidate: ‘ProtoSortedTree::Item* ProtoSortedTree::Iterator::PeekPrevItem()’ <near match>
                 Item* PeekPrevItem()
                       ^~~~~~~~~~~~
../protolib/include/protoTree.h:566:23: note:   passing ‘const ProtoSortedTreeTemplate<ITEM_TYPE>::Iterator*’ as ‘this’ argument discards qualifiers
../protolib/include/protoTree.h: In member function ‘ITEM_TYPE* ProtoSortedTreeTemplate<ITEM_TYPE>::Iterator::PeekNextItem() const’:
../protolib/include/protoTree.h:669:93: error: no matching function for call to ‘ProtoSortedTreeTemplate<ITEM_TYPE>::Iterator::PeekNextItem() const’
                     {return static_cast<ITEM_TYPE*>(ProtoSortedTree::Iterator::PeekNextItem());}
                                                                                             ^
../protolib/include/protoTree.h:564:23: note: candidate: ‘ProtoSortedTree::Item* ProtoSortedTree::Iterator::PeekNextItem()’ <near match>
                 Item* PeekNextItem()
                       ^~~~~~~~~~~~
../protolib/include/protoTree.h:564:23: note:   passing ‘const ProtoSortedTreeTemplate<ITEM_TYPE>::Iterator*’ as ‘this’ argument discards qualifiers
../protolib/include/protoTree.h:566:23: note:   passing ‘const ProtoSortedTreeTemplate<ITEM_TYPE>::Iterator*’ as ‘this’ argument discards qualifiers
../protolib/include/protoTree.h: In member function ‘ITEM_TYPE* ProtoSortedTreeTemplate<ITEM_TYPE>::Iterator::PeekNextItem() const’:
../protolib/include/protoTree.h:669:93: error: no matching function for call to ‘ProtoSortedTreeTemplate<ITEM_TYPE>::Iterator::PeekNextItem() const’
                     {return static_cast<ITEM_TYPE*>(ProtoSortedTree::Iterator::PeekNextItem());}
                                                                                             ^
../protolib/include/protoTree.h:564:23: note: candidate: ‘ProtoSortedTree::Item* ProtoSortedTree::Iterator::PeekNextItem()’ <near match>
                 Item* PeekNextItem()
                       ^~~~~~~~~~~~
../protolib/include/protoTree.h:564:23: note:   passing ‘const ProtoSortedTreeTemplate<ITEM_TYPE>::Iterator*’ as ‘this’ argument discards qualifiers
make: *** [Makefile.common:31: ../src/common/normMessage.o] Error 1
make: *** Waiting for unfinished jobs....
../src/common/normSession.cpp: In member function ‘void NormSession::SenderHandleCCFeedback(timeval, NormNodeId, UINT8, double, double, double, UINT16)’:
../src/common/normSession.cpp:3287:50: warning: enum constant in boolean context [-Wint-in-bool-context]
         bool haveRtt = (0 != (ccFlags && NormCC::RTT));
                                                  ^~~
make: *** [Makefile.common:31: ../src/common/normNode.o] Error 1
make: *** [Makefile.common:31: ../src/common/normObject.o] Error 1
make: *** [Makefile.common:31: ../src/common/normSession.o] Error 1

norm 1.5.9 build failure

👋 trying to build the latest release, but run into some build issue. The error log is as below:

build failure

==> ./waf install
Waf: Entering directory `/private/tmp/norm-20210130-97923-fcxwgj/norm-1.5.9/build'
Traceback (most recent call last):
  File "/private/tmp/norm-20210130-97923-fcxwgj/norm-1.5.9/.waf-2.0.19-1f3c580272b15a03d2566843c5fe872a/waflib/Scripting.py", line 119, in waf_entry_point
    run_commands()
  File "/private/tmp/norm-20210130-97923-fcxwgj/norm-1.5.9/.waf-2.0.19-1f3c580272b15a03d2566843c5fe872a/waflib/Scripting.py", line 182, in run_commands
    ctx=run_command(cmd_name)
  File "/private/tmp/norm-20210130-97923-fcxwgj/norm-1.5.9/.waf-2.0.19-1f3c580272b15a03d2566843c5fe872a/waflib/Scripting.py", line 173, in run_command
    ctx.execute()
  File "/private/tmp/norm-20210130-97923-fcxwgj/norm-1.5.9/.waf-2.0.19-1f3c580272b15a03d2566843c5fe872a/waflib/Scripting.py", line 413, in execute
    return execute_method(self)
  File "/private/tmp/norm-20210130-97923-fcxwgj/norm-1.5.9/.waf-2.0.19-1f3c580272b15a03d2566843c5fe872a/waflib/Build.py", line 93, in execute
    self.execute_build()
  File "/private/tmp/norm-20210130-97923-fcxwgj/norm-1.5.9/.waf-2.0.19-1f3c580272b15a03d2566843c5fe872a/waflib/Build.py", line 96, in execute_build
    self.recurse([self.run_dir])
  File "/private/tmp/norm-20210130-97923-fcxwgj/norm-1.5.9/.waf-2.0.19-1f3c580272b15a03d2566843c5fe872a/waflib/Context.py", line 133, in recurse
    user_function(self)
  File "/private/tmp/norm-20210130-97923-fcxwgj/norm-1.5.9/wscript", line 226, in build
    if ctx.options.enable_static_library:
AttributeError: Values instance has no attribute 'enable_static_library'

Full build log is in here, https://github.com/Homebrew/homebrew-core/runs/1800067295
relates to Homebrew/homebrew-core#70083

Developer guide is outdated

The developer guide (https://github.com/USNavalResearchLaboratory/norm/blob/master/doc/NormDeveloperGuide.pdf) is for version 1.5b5. There are some options that are in version 1.5.8 that do not appear to be in this develop guide. For example sync mode NORM_SYNC_STREAM is not described in the current version of the guide.

Suggestion: .NET NORM Extension

Suggest adding a .NET extension for NORM. This would provide a .NET wrapper so that .NET applications can access the NORM C API. It would be similar to the existing Java and Python extensions.

New release with cmake

Hi, I'm trying to get norm into conan to make it easier to consume. For that the cmake stuff helps a lot, thanks for letting that in. I would like to now consume norm in conan, but there is a strong preference for an official release.

Of course the previous release did not have cmake in there so to make this work I was wondering if another release would be possible?

Cheers!

Oh and a (possible) release is pending green build in conan, maybe I still have to do some cmake fixes conan-io/conan-center-index#4299

norm 1.5.9 fatal error with the python binding of session's method fileEnqueue

On python 2.7.18, with the interpreter, we use the same commands as the ones in the normFileSend.py example:

import pynorm
instance = pynorm.Instance()
session = instance.createSession("224.1.2.3", 6003)
session.setTxRate(256e6)
from random import randint
session.startSender(randint(0, 1000), 1024**2, 1400, 64, 16)
session.fileEnqueue("/home/titi/Documents/NORM/norm-1.5.9/VERSION.TXT", "VERSION.TXT")
Proto Fatal: NormObject::Open() error: object size exceeds FEC blocking and segmentation parameter capability
Proto Fatal: NormFileObject::Open() send object open error
Proto Fatal: NormSession::QueueTxFile() file open error
<pynorm.object.Object object at 0x7fe01b210d10>

2 compiler warnings seem to indicate errors

While compiling normApp.cpp

../src/common/normApp.cpp:617:20: warning: ‘void operator delete(void*, long unsigned int)’ called on pointer returned from a mismatched allocation function [-Wmismatched-new-delete]
  617 |             delete address;
      |                    ^~~~~~~
../src/common/normApp.cpp:607:39: note: returned from ‘void* operator new [](long unsigned int)’
  607 |         if (!(address = new char[len+1]))

and

../src/common/normPostProcess.cpp: In member function ‘void NormPostProcessor::GetCommand(char*, unsigned int)’:
../src/common/normPostProcess.cpp:51:20: warning: ‘char* strncpy(char*, const char*, size_t)’ output truncated before terminating nul copying 4 bytes from a string of the same length [-Wstringop-truncation]
   51 |             strncpy(buffer, "none", 4);
      |             ~~~~~~~^~~~~~~~~~~~~~~~~~~

both seem to indicate genuine errors.

The former should be changed to "delete[] address" (also in the lines 606 and 626 even though g++ did not notice).

About the latter, I am not sure as the enclosing function NormPostProcessor::GetCommand does not seem to be used so intentions are unclear. Unless it is there for future use, I think the safest would be to remove the function.

Segmentation fault (normObject.cpp)

I have made a small modification in the normDataSend.cpp example to continuously send new messages with random length. I did it simply, by adding a goto statement in the switch-case: NORM_TX_FLUSH_COMPLETED to repeat the whole process.

It seems working properly, however, after sending roughly around 50 messages, it throws a segmentation fault. Below is the stack, when the exception happens:

(gdb) where
#0  __memmove_avx_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:378
#1  0x00005555555df58d in NormDataObject::ReadSegment (this=0x555555a2e710, blockId=..., segmentId=0, buffer=0x555555948050 'a' <repeats 200 times>...) at ../src/common/normObject.cpp:2708
#2  0x00005555555dcdbb in NormObject::NextSenderMsg (this=0x555555a2e710, msg=0x555555948030) at ../src/common/normObject.cpp:1999
#3  0x00005555555eff00 in NormSession::Serve (this=0x555555916fa0) at ../src/common/normSession.cpp:1312
#4  0x00005555555fb985 in NormSession::OnTxTimeout (this=0x555555916fa0) at ../src/common/normSession.cpp:4848
#5  0x0000555555601fe8 in ProtoTimer::OLD_LISTENER_TYPE<NormSession>::old_on_timeout (this=0x555555915460, theTimer=...) at ../protolib/include/protoTimer.h:292
#6  0x000055555559d734 in ProtoTimer::OldListener::on_timeout (this=0x555555915460, theTimer=...) at ../protolib/include/protoTimer.h:266
#7  0x000055555559d667 in ProtoTimer::DoTimeout (this=0x555555916fb0) at ../protolib/include/protoTimer.h:206
#8  0x000055555559c428 in ProtoTimerMgr::OnSystemTimeout (this=0x555555914998) at ../protolib/src/common/protoTimer.cpp:180
#9  0x00005555555723d5 in ProtoDispatcher::Dispatch (this=0x555555914998) at ../protolib/src/common/protoDispatcher.cpp:1943
#10 0x00005555555708bc in ProtoDispatcher::Run (this=0x555555914998, oneShot=false) at ../protolib/src/common/protoDispatcher.cpp:989
#11 0x0000555555570aad in ProtoDispatcher::DoThreadStart (param=0x555555914998) at ../protolib/src/common/protoDispatcher.cpp:1060
#12 0x00007ffff7bbb6db in start_thread (arg=0x7ffff6e83700) at pthread_create.c:463
#13 0x00007ffff6fa561f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

PROTO_ADDR_BROADCAST not declared

When trying to build NORM from the latest commit (norm and Protolib) I am getting this error:

../src/common/normSession.cpp:5110:20: error: use of undeclared identifier 'PROTO_ADDR_BROADCAST' etherDst = PROTO_ADDR_BROADCAST;
And yes, there is no such definition in the source code.
I re-ran waf configure --build-java but no changes.
My system macOS 10.15.5 with the latest Xcode installed.

Suggestion: Make pynorm work with Python3

As https://www.python.org/doc/sunset-python-2/ , hope for supporting python3

waf build error

$ ./waf -v --targets=*
Waf: Entering directory `/home/xxx/norm/build'
[ 98/140] Linking build/libnorm.so
12:56:34 runner ['/opt/rh/devtoolset-7/root/usr/bin/g++', '-shared', '-Wl,-h,libnorm.so.1', 'src/common/galois.cpp.2.o', 'src/common/normApi.cpp.2.o', 'src/common/normEncoder.cpp.2.o', 'src/common/normEncoderMDP.cpp.2.o', 'src/common/normEncoderRS16.cpp.2.o', 'src/common/normEncoderRS8.cpp.2.o', 'src/common/normFile.cpp.2.o', 'src/common/normMessage.cpp.2.o', 'src/common/normNode.cpp.2.o', 'src/common/normObject.cpp.2.o', 'src/common/normSegment.cpp.2.o', 'src/common/normSession.cpp.2.o', '-o/home/xxx/norm/build/libnorm.so', '-Wl,-Bstatic', '-Lprotolib', '-lprotolib', '-lprotolib', '-Wl,-Bdynamic', '-lpthread', '-lnetfilter_queue']
[ 99/140] Compiling src/unix/unixPostProcess.cpp
12:56:34 runner ['/opt/rh/devtoolset-7/root/usr/bin/g++', '-fPIC', '-O3', '-fvisibility=hidden', '-Wno-attributes', '-Iprotolib/include', '-I../protolib/include', '-Iprotolib/include/unix', '-I../protolib/include/unix', '-I/usr/include/libxml2', '-DUNIX', '-DHAVE_DIRFD', '-DHAVE_IPV6', '-DHAVE_ASSERT', '-DHAVE_GETLOGIN', '-DLINUX', '-DHAVE_LOCKF', '-D_FILE_OFFSET_BITS=64', '-DHAVE_OLD_SIGNALHANDLER', '-DNO_SCM_RIGHTS', '-DHAVE_SCHED', '-DUSE_TIMERFD', '-DUSE_EVENTFD', '-DHAVE_PSELECT', '-DUSE_SELECT', '-DNDEBUG', '-DPROTO_DEBUG', '../src/unix/unixPostProcess.cpp', '-c', '-o/home/xxx/norm/build/src/unix/unixPostProcess.cpp.4.o']
[100/140] Compiling src/common/normPostProcess.cpp
12:56:34 runner ['/opt/rh/devtoolset-7/root/usr/bin/g++', '-fPIC', '-O3', '-fvisibility=hidden', '-Wno-attributes', '-Iprotolib/include', '-I../protolib/include', '-Iprotolib/include/unix', '-I../protolib/include/unix', '-I/usr/include/libxml2', '-DUNIX', '-DHAVE_DIRFD', '-DHAVE_IPV6', '-DHAVE_ASSERT', '-DHAVE_GETLOGIN', '-DLINUX', '-DHAVE_LOCKF', '-D_FILE_OFFSET_BITS=64', '-DHAVE_OLD_SIGNALHANDLER', '-DNO_SCM_RIGHTS', '-DHAVE_SCHED', '-DUSE_TIMERFD', '-DUSE_EVENTFD', '-DHAVE_PSELECT', '-DUSE_SELECT', '-DNDEBUG', '-DPROTO_DEBUG', '../src/common/normPostProcess.cpp', '-c', '-o/home/xxx/norm/build/src/common/normPostProcess.cpp.4.o']
../src/unix/unixPostProcess.cpp:44:10: fatal error: normPostProcess.h: No such file or directory
 #include "normPostProcess.h"
          ^~~~~~~~~~~~~~~~~~~
compilation terminated.

../src/common/normPostProcess.cpp:1:10: fatal error: normPostProcess.h: No such file or directory
 #include "normPostProcess.h"
          ^~~~~~~~~~~~~~~~~~~
compilation terminated.

Waf: Leaving directory `/home/xxx/norm/build'
Build failed
 -> task in 'normapp' failed with exit status 1:
        {task 139753374494544: cxx unixPostProcess.cpp -> unixPostProcess.cpp.4.o}
['/opt/rh/devtoolset-7/root/usr/bin/g++', '-fPIC', '-O3', '-fvisibility=hidden', '-Wno-attributes', '-Iprotolib/include', '-I../protolib/include', '-Iprotolib/include/unix', '-I../protolib/include/unix', '-I/usr/include/libxml2', '-DUNIX', '-DHAVE_DIRFD', '-DHAVE_IPV6', '-DHAVE_ASSERT', '-DHAVE_GETLOGIN', '-DLINUX', '-DHAVE_LOCKF', '-D_FILE_OFFSET_BITS=64', '-DHAVE_OLD_SIGNALHANDLER', '-DNO_SCM_RIGHTS', '-DHAVE_SCHED', '-DUSE_TIMERFD', '-DUSE_EVENTFD', '-DHAVE_PSELECT', '-DUSE_SELECT', '-DNDEBUG', '-DPROTO_DEBUG', '../src/unix/unixPostProcess.cpp', '-c', '-o/home/xxx/norm/build/src/unix/unixPostProcess.cpp.4.o']
 -> task in 'normapp' failed with exit status 1:
        {task 139753374494304: cxx normPostProcess.cpp -> normPostProcess.cpp.4.o}
['/opt/rh/devtoolset-7/root/usr/bin/g++', '-fPIC', '-O3', '-fvisibility=hidden', '-Wno-attributes', '-Iprotolib/include', '-I../protolib/include', '-Iprotolib/include/unix', '-I../protolib/include/unix', '-I/usr/include/libxml2', '-DUNIX', '-DHAVE_DIRFD', '-DHAVE_IPV6', '-DHAVE_ASSERT', '-DHAVE_GETLOGIN', '-DLINUX', '-DHAVE_LOCKF', '-D_FILE_OFFSET_BITS=64', '-DHAVE_OLD_SIGNALHANDLER', '-DNO_SCM_RIGHTS', '-DHAVE_SCHED', '-DUSE_TIMERFD', '-DUSE_EVENTFD', '-DHAVE_PSELECT', '-DUSE_SELECT', '-DNDEBUG', '-DPROTO_DEBUG', '../src/common/normPostProcess.cpp', '-c', '-o/home/xxx/norm/build/src/common/normPostProcess.cpp.4.o']

debugStats.py: No module named 'pynorm.extra.pipeparser'

In examples\python\debugStats.py

from pynorm.extra.pipeparser import PipeParser

Suggestion: Use Python enum in pynorm/constants.py

for Python2, use https://pypi.org/project/enum34/ instead.

the fairness among multiple sender sessions.

In the following code, I create two sessions in one process, only session s1 keeps send data:

import ipaddress
from random import randint
import datetime
import pynorm 

def create_Session(instance:pynorm.Instance, destAddr:str, destPort:int, localAddr:str, localPort:int=0):
    session = instance.createSession(destAddr, destPort,  localId=ipaddress.IPv4Address(localAddr)._ip )
    session.setTxPort(localPort,txBindAddr=localAddr) #
    session.setTxOnly(txOnly=True) 
    #session.setTxRate(256e10)
    session.setCongestionControl(ccEnable=True)
    session.startSender(randint(0, 1000), 100*1024*1024, segmentSize=1400, blockSize=128, numParity=0) 
    session.setGroupSize(4)
    return session 
    
    
if __name__ == '__main__':
    instance = pynorm.Instance()
    instance.setDebugLevel(level=1)
    
    localAddr = "10.1.1.1"
    destPort =6003
    data = b'xxxxxxxxxxxxxxx'*1024*1024
    #opts.address
    s1 = create_Session(instance, destAddr='224.1.2.3'  , destPort=destPort, localAddr=localAddr, localPort=0)
    s1.name='s1'
    s1.dataEnqueue(data= data )
    
    s2 = create_Session(instance, destAddr='224.1.2.4' , destPort=destPort, localAddr=localAddr, localPort=0)
    s2.name='s2'
    s2.dataEnqueue(data= data )
    try:
        for event in instance:
            event:pynorm.event.Event
            e = str(event).strip("NORM_")
            print( datetime.datetime.now().strftime("%H:%M:%S.%f"), e, event.session.name)
            if e in ('TX_OBJECT_SENT' , 'TX_OBJECT_PURGED' , 'TX_QUEUE_EMPTY') :
                result = event.session.dataEnqueue(data= data )            
    except KeyboardInterrupt:
        pass

Output:

19:02:46.367202 GRTT_UPDATED s1
19:02:46.367202 GRTT_UPDATED s2
19:02:49.571540 CC_ACTIVE s1
19:02:49.571540 TX_RATE_CHANGED s1
19:02:49.571540 CC_ACTIVE s2
19:02:49.571540 TX_RATE_CHANGED s2
19:02:50.711149 GRTT_UPDATED s1
19:02:51.007191 GRTT_UPDATED s1
19:02:51.208529 GRTT_UPDATED s1
19:02:51.409058 GRTT_UPDATED s1
19:02:51.578669 GRTT_UPDATED s1
19:02:51.750514 GRTT_UPDATED s1
19:02:51.889442 GRTT_UPDATED s1
19:02:52.012103 GRTT_UPDATED s1
19:02:52.058247 GRTT_UPDATED s2
19:02:52.121027 GRTT_UPDATED s1
19:02:52.236193 GRTT_UPDATED s1
19:02:52.313101 GRTT_UPDATED s1
19:02:52.331226 GRTT_UPDATED s2
19:02:52.394557 GRTT_UPDATED s1
19:02:52.513325 GRTT_UPDATED s1
19:02:52.513325 TX_OBJECT_SENT s1
19:02:52.516415 TX_QUEUE_EMPTY s1
19:02:52.552606 GRTT_UPDATED s2
19:02:52.563738 GRTT_UPDATED s1
19:02:52.612012 GRTT_UPDATED s1
19:02:52.681224 GRTT_UPDATED s1
19:02:52.912325 TX_OBJECT_SENT s1
19:02:53.004779 GRTT_UPDATED s1
19:02:53.022583 GRTT_UPDATED s1
19:02:53.063095 GRTT_UPDATED s1
19:02:53.154112 GRTT_UPDATED s1
19:02:53.185314 GRTT_UPDATED s1
19:02:53.211547 GRTT_UPDATED s1
19:02:53.345550 TX_OBJECT_SENT s1
19:02:53.347252 GRTT_UPDATED s1
19:02:53.349203 GRTT_UPDATED s1
19:02:53.352201 GRTT_UPDATED s1
19:02:53.358234 GRTT_UPDATED s1
19:02:53.368854 GRTT_UPDATED s1
19:02:53.389490 GRTT_UPDATED s1
19:02:53.621008 TX_OBJECT_SENT s1
19:02:53.640529 TX_QUEUE_EMPTY s1
19:02:53.885777 GRTT_UPDATED s1
19:02:53.916294 GRTT_UPDATED s2
19:02:53.975162 GRTT_UPDATED s2
19:02:53.987099 GRTT_UPDATED s1
19:02:53.998778 GRTT_UPDATED s1
19:02:54.009587 GRTT_UPDATED s1
19:02:54.027158 GRTT_UPDATED s1
19:02:54.054362 GRTT_UPDATED s1
19:02:54.168584 TX_OBJECT_SENT s1
19:02:54.193041 TX_QUEUE_EMPTY s1
19:02:54.305678 GRTT_UPDATED s1
19:02:54.346052 GRTT_UPDATED s1
19:02:54.495377 GRTT_UPDATED s1
19:02:54.562085 GRTT_UPDATED s1
19:02:54.625685 GRTT_UPDATED s1
19:02:54.755185 TX_OBJECT_SENT s1
19:02:54.769249 GRTT_UPDATED s1
19:02:54.866510 GRTT_UPDATED s1
19:02:54.939676 GRTT_UPDATED s1
19:02:55.045133 GRTT_UPDATED s1
19:02:55.045133 TX_OBJECT_SENT s1
19:02:55.046137 TX_QUEUE_EMPTY s1
19:02:55.046137 GRTT_UPDATED s1
19:02:55.046137 TX_OBJECT_PURGED s1
19:02:55.046137 GRTT_UPDATED s1
19:02:55.047138 GRTT_UPDATED s1
19:02:55.047138 GRTT_UPDATED s1
19:02:55.047138 GRTT_UPDATED s1
19:02:55.047138 TX_OBJECT_SENT s1
19:02:55.047138 GRTT_UPDATED s1
19:02:55.047138 GRTT_UPDATED s1
19:02:55.047138 TX_OBJECT_SENT s1
19:02:55.047138 TX_QUEUE_EMPTY s1
19:02:55.048410 TX_OBJECT_PURGED s1
19:02:55.048410 GRTT_UPDATED s1
19:02:55.048410 GRTT_UPDATED s1
19:02:55.048410 GRTT_UPDATED s1
19:02:55.048410 GRTT_UPDATED s1
19:02:55.048923 GRTT_UPDATED s1
19:02:55.048971 GRTT_UPDATED s1
19:02:55.048971 GRTT_UPDATED s2
19:02:55.048971 TX_OBJECT_SENT s1
19:02:55.048971 TX_QUEUE_EMPTY s1
19:02:55.048971 TX_OBJECT_PURGED s1
19:02:55.048971 TX_OBJECT_PURGED s1
19:02:55.048971 TX_OBJECT_PURGED s1
19:02:55.049935 TX_OBJECT_PURGED s1
19:02:55.049935 TX_OBJECT_PURGED s1
19:02:55.049935 TX_OBJECT_PURGED s1
19:02:55.208242 GRTT_UPDATED s1
19:02:55.272049 GRTT_UPDATED s1
19:02:55.423414 GRTT_UPDATED s1
19:02:55.618342 GRTT_UPDATED s1
19:02:55.641176 TX_QUEUE_EMPTY s1
19:02:56.150859 TX_OBJECT_SENT s1
19:02:56.315592 GRTT_UPDATED s1
19:02:56.326327 GRTT_UPDATED s1
19:02:56.342155 GRTT_UPDATED s1
19:02:56.391655 TX_OBJECT_SENT s1
19:02:56.831313 TX_OBJECT_PURGED s1
19:02:56.928489 GRTT_UPDATED s1
19:02:56.946144 GRTT_UPDATED s2
19:02:56.974019 GRTT_UPDATED s1
19:02:56.998517 GRTT_UPDATED s1
19:02:57.032450 GRTT_UPDATED s1
19:02:57.055062 GRTT_UPDATED s1
19:02:57.092916 TX_OBJECT_SENT s1
19:02:57.305245 GRTT_UPDATED s1
19:02:57.419702 GRTT_UPDATED s1
19:02:57.474402 TX_OBJECT_PURGED s1
19:02:57.566750 GRTT_UPDATED s1
19:02:57.580307 TX_OBJECT_SENT s1
19:02:57.766409 GRTT_UPDATED s1
19:02:57.786197 GRTT_UPDATED s2
19:02:57.814184 GRTT_UPDATED s2
19:02:57.825652 GRTT_UPDATED s1
19:02:57.841741 GRTT_UPDATED s1
19:02:57.928619 GRTT_UPDATED s1
19:02:57.959251 TX_OBJECT_PURGED s1
19:02:58.158236 GRTT_UPDATED s1
19:02:58.263278 GRTT_UPDATED s1
19:02:58.390227 TX_OBJECT_SENT s1
19:02:58.579250 GRTT_UPDATED s1
19:02:58.593876 GRTT_UPDATED s1
19:02:58.611716 GRTT_UPDATED s1
19:02:58.637049 GRTT_UPDATED s1
19:02:58.657264 GRTT_UPDATED s1
19:02:58.730318 TX_OBJECT_SENT s1
19:02:58.803523 TX_OBJECT_PURGED s1
19:02:58.917232 GRTT_UPDATED s1
19:02:58.938985 GRTT_UPDATED s1
19:02:58.950941 GRTT_UPDATED s1
19:02:58.960462 GRTT_UPDATED s1
19:02:58.971501 GRTT_UPDATED s1
19:02:58.994048 GRTT_UPDATED s1
19:02:59.055631 TX_OBJECT_SENT s1
19:02:59.153415 TX_OBJECT_PURGED s1
19:02:59.627289 GRTT_UPDATED s1
19:02:59.766426 GRTT_UPDATED s1
19:02:59.767565 GRTT_UPDATED s1
19:02:59.785161 GRTT_UPDATED s1
19:02:59.805698 GRTT_UPDATED s1
19:02:59.851661 GRTT_UPDATED s1
19:02:59.988465 GRTT_UPDATED s1
19:03:00.006055 GRTT_UPDATED s1
19:03:00.122846 TX_OBJECT_SENT s1
19:03:00.316230 GRTT_UPDATED s1
19:03:00.324909 GRTT_UPDATED s1
19:03:00.340978 TX_OBJECT_PURGED s1
19:03:00.414464 GRTT_UPDATED s1
19:03:00.442500 GRTT_UPDATED s1
19:03:00.487322 GRTT_UPDATED s1
19:03:00.519405 TX_OBJECT_SENT s2
19:03:00.647415 TX_QUEUE_EMPTY s2
19:03:00.702349 TX_OBJECT_PURGED s1
19:03:01.134269 GRTT_UPDATED s1
19:03:01.153242 TX_OBJECT_SENT s1
19:03:01.300629 GRTT_UPDATED s1
19:03:01.330173 GRTT_UPDATED s1
19:03:01.338429 GRTT_UPDATED s1
19:03:01.349977 GRTT_UPDATED s1
19:03:01.360303 GRTT_UPDATED s1
19:03:01.384839 GRTT_UPDATED s1
19:03:01.396490 GRTT_UPDATED s1
19:03:01.410553 GRTT_UPDATED s1
19:03:01.446692 TX_OBJECT_SENT s1
19:03:01.641648 TX_OBJECT_PURGED s1
19:03:01.885609 GRTT_UPDATED s1
19:03:01.996293 GRTT_UPDATED s1
19:03:02.001411 GRTT_UPDATED s1
19:03:02.005331 GRTT_UPDATED s1
19:03:02.022968 GRTT_UPDATED s1
19:03:02.044058 GRTT_UPDATED s1
19:03:02.064358 GRTT_UPDATED s1
19:03:02.080898 GRTT_UPDATED s1
19:03:02.109323 GRTT_UPDATED s1
19:03:02.192692 TX_OBJECT_SENT s1
19:03:02.193687 GRTT_UPDATED s1
19:03:02.193687 GRTT_UPDATED s1
19:03:02.193687 GRTT_UPDATED s1
19:03:02.193687 GRTT_UPDATED s1
19:03:02.193687 GRTT_UPDATED s1
19:03:02.193687 GRTT_UPDATED s1
19:03:02.193687 GRTT_UPDATED s1
19:03:02.193687 TX_OBJECT_PURGED s1
19:03:02.194809 TX_OBJECT_SENT s1

function `NormChangeDestination` missed in pynorm and NormDeveloperGuide

as topic

function `setTxOnly` missed in pynorm/session.py

As topic

    def setTxOnly(self, txOnly:bool=False, connectToSessionAddress:bool=False):
        '''
            void NormSetTxOnly(NormSessionHandle sessionHandle,
                       bool              txOnly,
                       bool              connectToSessionAddress)
        '''
        libnorm.NormSetTxOnly(self, txOnly, connectToSessionAddress)

Receiving File and writing to mounted share is slow

I'm using norm to send files unidirectionaly/silent to a receiver.
Our receiver is a .NET Application which uses a SWIG wrapper for norm, but we are able to reproduce the problem with the example normApp.

Normally we just receive the files an use a local directory as a cache dir. This works as expected.
But now we tried to use a mounted samba share and this leads to very regular failures.
norm receives the files on interface1 and the the samba share is connected to interface2. Those are physical interfaces.

EDIT
We try to send 20 files between 250 and 1000Mbyte.
It does work when we send smaller files
END EDIT
We already restricted sending to 250Mbit (which is rather slow, we are able to get higher when using a local directory)

What I was able to observe via iftop is, that the data is received on interface1 with about 250Mbit/s but iftop on interface2 shows only a speed of about 130Mbit/s. After some time there is a sudden spike to 500-800Mbit.

We did also some tests with udpcast, and although it works a bit different, iftop on interface2 shows a continous speed of about 500-800Mbit and doesn't seem to fail at all.
There are other factors our udpcast application does different, but still the difference in speed is eye-catching.

I also tested the smb mount with dd which showed good speed in line with udpcast.

This lead me here, although I'm not sure yet if it is a problem with norm, protolib or something different. (currently it seems to point to protolib).

I will try to investigate the source code, but since I'm not a C++ developer, I wonder if you might have some ideas or insights.
Any pointers where to look?
Maybe some block size or caching issue when writing the file?

Thanks

tl;dr

norm wrapper and the normApp write with about 130Mbit/s to a samba share and fails (probably because buffer are getting filled because we send with 250Mbit/s)
if a local directory is used, it works.
udpcast writes consistently with 500-800Mbit.

NormStreamSend Java Example Does Not Compile

The NormStreamSend Java example does not compile.

Used the Waf build tool to configure to build the Java extension:
./waf configure --build-java
Command was successful:
'configure' finished successfully (5.141s)
Built the Java extension using the Waf build tool:
./waf
Command was successful:
'build' finished successfully (58.831s)
Switched to the Java examples directory and attempted to compile the Java examples based on the README.TXT:
javac -cp ../../build/norm.jar *.java
Received the following error about NormStreamSend:

NormStreamSend.java:49: error: incompatible types: possible lossy conversion from int to short
                                                                SEGMENT_SIZE, BLOCK_SIZE, PARITY_SEGMENTS);
                                                                              ^
Note: Some messages have been simplified; recompile with -Xdiags:verbose to get full output
1 error

Changed NormStreamSend BLOCK_SIZE to short on line 13:
static final short BLOCK_SIZE = 64;
Attempted to compile the Java examples again:
javac -cp ../../build/norm.jar *.java
Received the following error about NormStreamSend:

NormStreamSend.java:49: error: incompatible types: possible lossy conversion from int to short
                                                                SEGMENT_SIZE, BLOCK_SIZE, PARITY_SEGMENTS);
                                                                                          ^
Note: Some messages have been simplified; recompile with -Xdiags:verbose to get full output
1 error

Changed NormStreamSend PARITY_SEGMENTS to short on line 14:
static final short PARITY_SEGMENTS = 16;
Attempted to compile the Java examples again:
javac -cp ../../build/norm.jar *.java
Build was successful.

galois.h not prefixed with norm

This could be an issue of someone already has a galois.h in their include folder. I have not encountered this.

norm 1.5.9 python binding of setRxPortReuse session's method seems not to work

The call of the setRxPortReuse with one boolean argument works with standard C++ lib but not with python: i tried what follows with python 2.7.18 interpreter:

import pynorm
instance = pynorm.Instance()
session = instance.createSession("224.1.2.3", 6003)
session.setRxPortReuse(True)
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python2.7/dist-packages/pynorm/session.py", line 48, in setRxPortReuse
libnorm.NormSetRxPortReuse(self, enable, bindToSessionAddr)
TypeError: this function takes at least 5 arguments (3 given)

norm 1.5.9 python binding failure

On ubuntu 20.04 with python 2.7.18: I get this:

import pynorm
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python2.7/dist-packages/pynorm/init.py", line 6, in
from pynorm.instance import Instance
File "/usr/local/lib/python2.7/dist-packages/pynorm/instance.py", line 17, in
import pynorm.constants as c
File "/usr/local/lib/python2.7/dist-packages/pynorm/constants.py", line 10, in
from pynorm.core import libnorm
File "/usr/local/lib/python2.7/dist-packages/pynorm/core.py", line 510, in
libnorm = get_libnorm()
File "/usr/local/lib/python2.7/dist-packages/pynorm/core.py", line 504, in get_libnorm
libnorm.NormCountCompletedObjects.restype = ctypes.c_uint32
File "/usr/lib/python2.7/ctypes/init.py", line 379, in getattr
func = self.getitem(name)
File "/usr/lib/python2.7/ctypes/init.py", line 384, in getitem
func = self._FuncPtr((name_or_ordinal, self))
AttributeError: /home/titi/Documents/NORM/norm-1.5.9/build/libnorm.so: undefined symbol: NormCountCompletedObjects

config.log

ProtoTree x->right ==x Caused infinite loop

Call Stack：

 	Norm.dll!ProtoTree::Bit(const char * key=0x000001db527ca670, unsigned int keysize=96, unsigned int index=64, ProtoTree::Endian keyEndian=ENDIAN_BIG) Line 232	C++
>	Norm.dll!ProtoTree::Remove(ProtoTree::Item & item={...}) Line 459	C++
 	[Inline Frame] Norm.dll!ProtoTreeTemplate<ProtoSortedTree::Item>::Remove(ProtoSortedTree::Item &) Line 354	C++
 	Norm.dll!ProtoSortedTree::Remove(ProtoSortedTree::Item & item={...}) Line 1640	C++
 	Norm.dll!ProtoTimerMgr::RemoveLongTimer(ProtoTimer & theTimer={...}) Line 508	C++
 	Norm.dll!ProtoTimerMgr::OnPulseTimeout(ProtoTimer & __formal) Line 218	C++
 	[Inline Frame] Norm.dll!ProtoTimer::DoTimeout() Line 206	C++
 	Norm.dll!ProtoTimerMgr::OnSystemTimeout() Line 181	C++
 	Norm.dll!ProtoDispatcher::Dispatch() Line 2702	C++
 	Norm.dll!ProtoDispatcher::Run(bool oneShot) Line 1003	C++
 	Norm.dll!ProtoDispatcher::DoThreadStart(void * param=0x000001db52299b18) Line 1061	C++

Code：ProtoTree::Bit

        do                      
        {     
            q = x;              
            if (Bit(key, keysize, x->bit, keyEndian))
                x = x->right; // !!! x == x->right  !!!
            else                
                x = x->left;    
        } while (x != &item);

Unable to build for Android

I am currently trying to build norm to use in an android project. I have tried both the gradle and ant method listed in the repo. The issue with the gradle directory is that it seems to be an empty gradle project. On the other hand the ant method under the android-ndk directory is not compatible with newer android studio versions. When I try to setup the project I continue to receive an error because the android tool is deprecated and doesn't include the options required by the README.

Are there any other methods to compile this with a modern gradle dependent android apk?

Sending an empty file (0 Bytes) does not trigger TX_OBJECT_SENT event

If I send an empty file (e.g. with the normFileSend or normCast example) the file is sent and received correctly, but the TX_OBJECT_SENT event is not triggered. The TX_QUEUE_EMPTY event is raised correctly after the file is sent.

I tried to dig into the code, but I could not found the right place to fix the problem.

Error when interface is not working

When I use NORM on Android over an WiFi and the connection goes down(not disabling the interface) I see the following error in the log, but I don't get any event to handle the issue:

2020-01-29 12:58:01.749 ? W/protolib: NormSession::SendMessage() sendto(224.1.2.3/4102) warning: Invalid argument
2020-01-29 12:58:01.752 ? E/protolib: ProtoSocket::SendTo() sendto() error: Invalid argument

The error is shown in an endless loop and I can't handle it. Is there any way to determine if NORM can send the data correctly?

add arguments `txBindAddress` back to pynorm Session.setTxPort

If a host has multi network interfaces, and one interface has many ip addresses, set txBindAddress is the easy way to make things work.

    def setTxPort(self, port:int, addr:str="" ):
        if addr:
            libnorm.NormSetTxPort(self, port,False, addr.encode() )
        else:
            libnorm.NormSetTxPort(self, port)

NormRestartInstance cause ASSERT INVALID_DESCRIPTOR error

test code:

    NormInstanceHandle instance = NormCreateInstance();
    NormRestartInstance(instance);

assert error position: protolib\src\common\protoDispatcher.cpp(2192)

bool ProtoDispatcher::InstallBreak()
{
    ASSERT(INVALID_DESCRIPTOR == break_event.GetDescriptor());

Configure error

Fresh install. Already installed protolib.

[xxx@localhost norm]$ ./waf configure
Cannot read the folder '/home/xxx/norm/protolib'

Getting NORM_REMOTE_SENDER_ADDRESS event

Good day NRL team.

I'm running into an issue with NormFileSend.cpp and NormFileRecv.cpp examples. I'm trying to send different files using few containerized NormFileSend instances to a single NormFIleRecv container instance. I do this in order, ie not simultaneously, back to back transfer. It works mostly fine, except every few sends I start getting NORM_REMOTE_SENDER_ADDRESS events continuously and sometimes it clears out, sometimes it gets stuck until a timeout and then sends/receive resumes until another time it gets stuck. Before I was getting NORM_REMOTE_SENDER_RESET event as well, but I fixed that with a static instanceID. I was going to try to use NormServer and NormClient examples as it more aligns with multiple transfers, many to one / one to many, but saw the comment that its not complete yet. Is there a way I can alleviate NORM_REMOTE_SENDER_ADDRESS event? My guess is that since Multiple senders with different IP addresses are sending, the receiver gets this event, but shouldn't destruction of the instance and closeout of the session account for this? It almost seems that some information from previous transfer remains when a new instance is created and they overlap?
Any advise, examples would be greatly appreciated.
Thank you!

Unable to send 10mb file

I am attempting to send a 10mb file from my MacBook to my Raspberry Pi as the receiver. However, I get an error:

Proto Error: ProtoSocket::SendTo() sendto() error: Message too long

I attempted to enable the fragmentation setting within norm but that didn't make a difference. I also altered my multicast MTU limit on both of my boxes as I read that there is a limit imposed by the system by default. Using norm am I able to send message and files over 1500 bytes? And if so, what do I have to enable with norm or with my box?

future support Raptor Q

NormObjectTable::Remove(NormObject): Assertion `__null != tree.Find(theObject->GetId().GetValuePtr(), 8sizeof(UINT16))' failed

Since the delay and async-behavior of high-level event process code and low-level NORM code.

User may call NormObjectCancel while the object already been NormSenderNode::AbortObject by NORM protocol like NormSenderNode::Sync(NormObjectId objectId), which caused object is removed twice in NormSenderNode.rx_table.

void NormSenderNode::DeleteObject(NormObject* obj)
{
    if (rx_table.Remove(obj))
    {
        rx_pending_mask.Unset(obj->GetId());
        obj->Close();
        obj->Release();
    }
}  // end NormSenderNode::DeleteObject()

404 page on project site

👋 just a reminder the site is currently showing 404 page. Thanks!

The Licence in LICENCE and Readme.md are different

Which one of the licences is the correct one?

Transfer Rate

Hi,
Recently I got to know norm and I'm giving it a try. Apparently it has a TCP-friendly congestion control. However, when I'm trying to use the normCast example, the sending rate doesn't go beyond 11.2 kbps. Is it normal? I'm running the commands in mininet. Here are the commands I'm using:

sender:

./normCast id 7 send u.pdf addr 172.16.0.100 cc

receiver:

./normCast id 10 recv ./

part of the debugging info in the sender:

Proto Info: ProtoDebug>SetDebugLevel: debug level changed from 1 to 10
Proto Debug: NormSession::SetTxRateInternal() node>7 increased to new grtt to: 1.064000 sec
Proto Debug: SenderRateTracking time>1638473409.821098 rate>11.200000 rtt>0.001047 loss>0.000000
normCast: enqueued "u.pdf" for transmission ...
Proto Debug: SenderRateTracking time>1638473410.885716 rate>11.200000 rtt>0.001047 loss>0.000000
Proto Debug: SenderRateTracking time>1638473411.950401 rate>11.200000 rtt>0.001047 loss>0.000000
Proto Debug: SenderRateTracking time>1638473413.013693 rate>11.200000 rtt>0.001047 loss>0.000000
Proto Debug: SenderRateTracking time>1638473414.078423 rate>11.200000 rtt>0.001047 loss>0.000000
Proto Debug: SenderRateTracking time>1638473415.141370 rate>11.200000 rtt>0.001047 loss>0.000000
Proto Debug: SenderRateTracking time>1638473416.205700 rate>11.200000 rtt>0.001047 loss>0.000000
Proto Debug: SenderRateTracking time>1638473417.270224 rate>11.200000 rtt>0.001047 loss>0.000000
Proto Debug: SenderRateTracking time>1638473418.333997 rate>11.200000 rtt>0.001047 loss>0.000000
Proto Debug: SenderRateTracking time>1638473419.398407 rate>11.200000 rtt>0.001047 loss>0.000000
Proto Info: REPORT time>19:30:19.774828 node>7 ***************************************
Proto Info: Local status:
Proto Info:    txRate>   11.200 kbps sentRate>   11.684 grtt>1.064000
Proto Info: ***************************************************************************

normCast not support dir?

commond args:
id 172042175 send E:\PythonPrj\testFiles100K addr 224.1.2.4/6003 txAddr 10.65.39.191/8002 segment 1400 block 128 parity 4 auto 1 rate 5000000 buffer 50000000 debug 3

stack:

	normCast.exe!NormCaster::EnqueueFileObject() Line 608	C++
 	normCast.exe!NormCaster::SendFiles() Line 559	C++
 	normCast.exe!main(int argc, char * * argv) Line 1578	C++

error:

normCast error: transmit file path ""  exceeds NORM segment size limit!