Giter VIP home page Giter VIP logo

gmsec_api's Introduction

This README file provides general information regarding the GMSEC API, its
supported middleware, and example programs.


1. DISTRIBUTIONS

   The GMSEC API is available in two different distributions (Open-Source and
   Government version) for the following OSes:

      * Windows 10
      * Windows 11
      * RHEL 8
      * Ubuntu 20.04 LTS
      * Ubuntu 20.04 ARM/Aarch64 LTS
      * MacOS

   Binaries and source code for the Open-Source version is available on NASA
   GitHub. See here for further details: https://github.com/nasa/GMSEC_API

   Instructions for building the Open-Source code can be found in the
   INSTALL.txt file located at the root folder level where the GMSEC API project
   is installed.

   As for the Government version, it is only available in binary format (no
   source code), and more importantly it is only available by request from GMSEC
   by those supporting a US Government contract. This version contains the
   Compat-C2 interface (for NSS message encrption); for users that do not
   require this feature, then it is recommended to use the Open-Source
   distribution.



2. DOCUMENTATION

   The GMSEC API includes documentation is the docs directory which is located
   in the base directory where the GMSEC API has been installed.

   There the following documents exist:

      * GMSEC API Installation and Configuration Guide
      * GMSEC API SRS
      * GMSEC API User's Guide
      * GMSEC API VDD
      * GMSEC API CompatC2 Installation and Configuration Guide [see note below]
      * NASA C2MS Addendum [see note below]

   In addition, Doxygen style documentation for the GMSEC API may be found in
   the docs/manual folder. Open index.html for the appropriate language binding 
   using a web browser.

   Note:
   These documents are *not* included with the Open-Source release of the GMSEC 
   API.



3. SOFTWARE INSTALLATION and SYSTEM CONFIGURATION

   The GMSEC API can be installed anywhere on a system, however it is
   recommended to choose a folder that does not contain whitespace. Special user
   accounts are not necessary to use the GMSEC API, nor are admin privileges.

   Once the software is installed, it is recommended to reference its location
   using the GMSEC_API_HOME environment variable, and then to use this to
   configure other environment variables. For example:

   On Linux:

      export GMSEC_API_HOME=/opt/GMSEC/GMSEC_API

      export LD_LIBRARY_PATH=$GMSEC_API_HOME/bin:$LD_LIBRARY_PATH

      export PATH=$PATH:$GMSEC_API_HOME/bin

      export CLASSPATH=$GMSEC_API_HOME/bin/gmsecapi.jar:$CLASSPATH


   On Windows:

      set GMSEC_API_HOME=C:\GMSEC\GMSEC_API

      set PATH=%PATH%;%GMSEC_API_HOME%\bin

      set CLASSPATH=%GMSEC_API_HOME%\bin\gmsecapi.jar;%CLASSPATH%



4. MIDDLEWARE BROKERS

   The GMSEC API supports various middleware brokers; these include:

      * Apache ActiveMQ
      * Apache Artemis
      * Bolt
      * IBM MQ
      * Message Bus
      * OpenDDS
      * RabbitMQ (using AMQP 1.0)
      * ZeroMQ

   Information on acquiring, installing, and running a middleware broker may be
   found in the GMSEC API Installation and Configuration Guide.



5. MIDDLEWARE CLIENT LIBRARIES

   To interact with a middleware broker, the GMSEC API will need to rely on
   vendor-specific client libraries. Typically these are downloaded from the
   respective vendor. The exception is with Bolt and Message Bus whose libraries
   are included with the GMSEC API distribution.

   The Apache ActiveMQ client libraries (which is also used for Artemis) can be
   obtained by request from the GMSEC team. Alternatively, the user can build
   these themselves from source code.

   Once installed, middleware client libraries should be referenced; for
   example:

   On Linux:

      export LD_LIBRARY_PATH=/path/to/client/lib:$LD_LIBRARY_PATH


   On Windows:

      set PATH=%PATH%;C:\path\to\client\lib



6. UTILITIES/TOOLS

   The GMSEC API provides a small suite of utility/tool applications. These
   include:

      * gmconfig_edit      # console app to assist creating a configuration file
      * gmhelp             # provides general middleware information
      * gmlog              # publishes a Log message
      * gmpub              # publishes a Heartbeat message
      * gmreq              # issues a Request Directive message
      * gmrpl              # issues a Response Directive message
      * gmsub              # subscribes to receive messages
      * msg_schema_ids     # lists available schema IDs
      * throughput_pub     # stress test that publishes messages
      * throughput_sub     # stress test that subscribes to receive messages



7. SUPPORTED PROGRAMMING LANGUAGE BINDINGS

   The GMSEC API provides support for multiple programming languages. These
   include:

      * C
      * C++
      * C#
      * Java
      * NodeJS
      * Perl
      * Python
      * Ruby

   Information regarding developing applications using these bindings can be
   found in the GMSEC API User's Guide.

   Notes:

      1. Mono is required to use C# under Linux and MacOS
      2. Strawberry Perl 5.32.1 is required use Perl under Windows
      3. Python 3.10.7 (or later 3.10.x) is required to use Python
      4. Ruby 3.0.2 is required to use Ruby under Windows
      5. Node 18.15.0 (or later) is required for NodeJS



8. EXAMPLE PROGRAMS

   The GMSEC API provides a suite of example programs located under the examples
   directory located at the base directory where the GMSEC API distribution is
   installed. Each examples directory contains a README.txt file with
   information on how to configure one's environment to run the example
   programs.

   Those that are required to be compiled/linked will need for the working
   system to have development tools installed. For Linux, this will, at a
   minimum, require GCC and Make. For Windows, Visual Studio 2019 or later will
   be required.



9. SUPPORT

   For GMSEC API and for basic middleware support, feel free to send queries to
   [email protected].


gmsec_api's People

Contributors

dmwhitne-583 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

gmsec_api's Issues

activemq-cpp wrapper ignores sec-ssl-truststore unless sec-ssl-truststore-key is set

When using a CA certificate file and activemq GMSEC_API ignores the CA certificate file unless sec-ssl-truststore-key is set to a non-empty string. See [1]. This is confusing for users since activemq-cpp does not have the capability to use a CA certificate file key. See [2].

Recommended fix is to pass along the sec-ssl-truststore value iff it is not empty, regardless of the value of sec-ssl-truststore-key.

[1] https://github.com/nasa/GMSEC_API/blob/master/wrapper/activemq/src/CMSConnection.cpp#L687
[2] https://github.com/apache/activemq-cpp/blob/d6f76ede90d21b7ee2f0b5d4648e440e66d63003/activemq-cpp/src/main/decaf/internal/net/ssl/openssl/OpenSSLContextSpi.cpp#L238

SIGSEGV in Message.getSubject()

As a Java developer I expect NASA's code to never segfault and kill the JVM. NullPointerExceptions are fine, segfaults are not.

Using GMSEC API version 4.6 I can produce a segfault every time by calling getSubject() on a received REPLY message. I can call getSubject() on received PUBLISH or REQUEST messages without any problems. Reproducer code and stack trace top are below.

I assume the problem is not actually in std::string::c_str(). My theory is that the issue is due to using reinterpret_cast<Message*>(jlong) in the native code on some long that is not the address of a message. How NASA's Java interface got an invalid pointer is beyond me.

Current thread (0x00007f08c0012800):  JavaThread "main" [_thread_in_native, id=289137, stack(0x00007f08c6ee0000,0x00007f08c6fe0000)]

siginfo: si_signo: 11 (SIGSEGV), si_code: 128 (SI_KERNEL), si_addr: 0x0000000000000000

Register to memory mapping:

RAX=0x771abb87bdb6526b is an unknown value
RBX={method} {0x00007f0878441d68} 'Message_GetSubject' '(JLgov/nasa/gsfc/gmsec/api/jni/JNIMessage;)Ljava/lang/String;' in 'gov/nasa/gsfc/gmsec/api/jni/gmsecJNI'
RCX=0x00007f08c6fdbeb0 is pointing into the stack for thread: 0x00007f08c0012800
RDX=0x00007f0828002930 is an unknown value
RSP=0x00007f08c6fdbc38 is pointing into the stack for thread: 0x00007f08c0012800
RBP=0x00007f08c6fdbc50 is pointing into the stack for thread: 0x00007f08c0012800
RSI=0x00007f08c6fdbea0 is pointing into the stack for thread: 0x00007f08c0012800
RDI=0x771abb87bdb6526b is an unknown value
R8 =0x00007f087850b928 is pointing into metadata
R9 =0x0000000000000009 is an unknown value
R10=0x00007f08b10183fb is at code_begin+859 in an Interpreter codelet
method entry point (kind = native)  [0x00007f08b10180a0, 0x00007f08b1018a40]  2464 bytes
R11=0x00007f08c7a2ca00: <offset 0xa2ca00> in /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so at 0x00007f08c7000000
R12=0x0000000000000000 is an unknown value
R13={method} {0x00007f0878441d68} 'Message_GetSubject' '(JLgov/nasa/gsfc/gmsec/api/jni/JNIMessage;)Ljava/lang/String;' in 'gov/nasa/gsfc/gmsec/api/jni/gmsecJNI'
R14=0x00007f08c6fdbec0 is pointing into the stack for thread: 0x00007f08c0012800
R15=0x00007f08c0012800 is a thread

Stack: [0x00007f08c6ee0000,0x00007f08c6fe0000],  sp=0x00007f08c6fdbc38,  free space=1007k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [libstdc++.so.6+0xf3c64]  std::string::c_str() const+0x4
C  [libGMSECAPI.so+0x314cea]  gmsec::api::Message::getSubject() const+0x1c
C  [libgmsec_jni.so+0x1083e3]  Java_gov_nasa_gsfc_gmsec_api_jni_gmsecJNI_Message_1GetSubject+0x6b
j  gov.nasa.gsfc.gmsec.api.jni.gmsecJNI.Message_GetSubject(JLgov/nasa/gsfc/gmsec/api/jni/JNIMessage;)Ljava/lang/String;+0
j  gov.nasa.gsfc.gmsec.api.jni.JNIMessage.getSubject()Ljava/lang/String;+5
j  gov.nasa.gsfc.gmsec.api.Message.getSubject()Ljava/lang/String;+4
public class SigsegvTest {


    private static BoltServer boltServer;

    @BeforeAll
    public static void setup() throws InterruptedException {
        // load GMSEC_API native code
        GmsecLoader.loadGmsec();
        // start bolt server
        boltServer = new BoltServer();
        new Thread(boltServer, "boltServer").start();
        boltServer.waitUntilStarted();
    }

    @AfterAll
    public static void cleanup() {
        boltServer.stop();
        GmsecLoader.cleanGmsec();
    }

    @TempDir
    Path schemaDir;

    @Test
    public void testSegfault() throws GMSEC_Exception, IOException, InterruptedException {
        // setup
        Log.registerHandler(new LogHandler() {
            @Override
            public void onMessage(LogEntry logEntry) {
                System.err.format("%s.%09d [%s] %s:%s %s\n",
                        logEntry.time.getSeconds(), logEntry.time.getNanoseconds(),
                        logEntry.level,
                        logEntry.fileName, logEntry.lineNumber, logEntry.message);
            }
        });
        Log.setReportingLevel(LogLevel.VERBOSE);
        Config config = new Config();
        config.addValue("connectiontype", "gmsec_bolt");
        config.addValue("server", "localhost");
        config.addValue("mw-multi-resp", "true");
        config.addValue("GMSEC-MSG-CONTENT-VALIDATE", "false");
        config.addValue("GMSEC-MSG-CONTENT-VALIDATE-ALL", "false");
        config.addValue("GMSEC-MSG-CONTENT-VALIDATE-SEND", "false");
        config.addValue("GMSEC-MSG-CONTENT-VALIDATE-RECV", "false");
        config.addValue("GMSEC-VALIDATION-LEVEL", "NO_ENFORCEMENT");
        config.addValue("GMSEC-SCHEMA-LEVEL", "0");
        config.addValue("GMSEC_LEGACY_SCHEMA_FILES", "true");
        config.addValue("GMSEC-SPECIFICATION-VERSION", "201600");
        // I wish I could turn off message validation entirely, but I can't
        // so fake a directory of schemas
        Files.createDirectories(schemaDir.resolve("2016.00"));
        try (Writer out = Files.newBufferedWriter(schemaDir.resolve("2016.00").resolve(".DIRECTORY.xml"))) {
            out.write("<DIRECTORY NAME=\"2016.00\"><SCHEMA NAME=\"HEADER\" LEVEL-0=\"GMSEC\" DEFINITION=\"\"/></DIRECTORY>");
        }
        try (Writer out = Files.newBufferedWriter(schemaDir.resolve("2016.00").resolve("GMSEC.HEADER.xml"))) {
            out.write("<SCHEMA NAME=\"GMSEC.HEADER\"></SCHEMA>");
        }
        config.addValue("GMSEC-SCHEMA-PATH", schemaDir.toAbsolutePath().toString());
        String subject = "GMSEC.1.2.3.4.5.6";
        List<Message> messages = new ArrayList<>();

        // subscribe
        final ConnectionManager cm1 = new ConnectionManager(config);
        cm1.initialize();
        cm1.startAutoDispatch();
        Thread.sleep(500);
        cm1.subscribe("GMSEC.+", new ConnectionManagerCallback() {
            @Override
            public void onMessage(ConnectionManager connectionManager, Message message) {
                // save request message
                messages.add(message);
                try {
                    if (message.getKind().equals(MessageKind.REQUEST)) {
                        // create and send two replies
                        Message reply = new Message(message.getSubject(), MessageKind.REPLY);
                        connectionManager.reply(message, reply);
                        /*
                        reply = new Message(message.getSubject(), MessageKind.REPLY);
                        connectionManager.reply(message, reply);
                         */
                    }
                } catch (GMSEC_Exception e) {
                    throw new RuntimeException(e);
                }
            }
        });
     
        // send request

        try {
            final ConnectionManager cm2 = new ConnectionManager(config);
            cm2.initialize();
            Thread.sleep(500);
            Message request = new Message(subject, MessageKind.REQUEST);
            cm2.request(
                    request,
                    60000,
                    new ConnectionManagerReplyCallback() {
                        @Override
                        public void onReply(ConnectionManager connectionManager,
                                            Message request,
                                            Message reply) {
                            // save message
                            messages.add(reply);
                        }

                        @Override
                        public void onEvent(ConnectionManager connectionManager,
                                            Status status,
                                            ConnectionEvent connectionEvent) {
                            System.err.println("" + status + " " + connectionEvent);
                        }
                    },
                    -1);

        } catch (GMSEC_Exception | InterruptedException e) {
            throw new RuntimeException(e);
        }

        // inelegant wait for messages to arrive
        Thread.sleep(5000);

        // verify saved messages
        // check waited long enough for the messages to arrive
        assertThat(messages.size(), is(2));
        // check received request message has a subject
        assertThat(messages.get(0).getSubject(), not(nullValue()));
        // check received reply message has a subject
        // this causes a SIGSEGV
        assertThat(messages.get(1).getSubject(), not(nullValue()));
    }

}

Deadlock publishing message

Came across this issue when attempting to make my application robust to the connection dropping out and reconnecting. Restarting the heartbeat thread and sending a log message that the connection was established deadlocked the application. May be related to API-5587, but I don't know the details of that issue.

Steps to reproduce:

  1. Create a ConnectionManager with an active bolt connection
  2. Register a handler for CONNECTION_SUCCESSFUL_EVENT events. The handler will start a new thread that publishes a message and then publish a message of its own.
  3. kill the bolt bus.
  4. restart the bolt bus

Expected result:

Step 4 triggers the handler registered in step 2 which causes two messages to be published on the bus.

Actual result:

Application deadlocks. Zero messages are published.

Analysis:

The thread dispatching the CONNECTION_SUCCESSFUL_EVENT holds the mutex in the bolt wrapper. (Shared.cpp line 81 and 111) Control passes to the event handler while that mutex is held (Shared.cpp line 146, line 89). This means that at this point only the thread handling the event is allowed to send messages. Hopefully it doesn't try to acquire any more locks... but it will. First the thread started by handler attempts to publish a message, acquiring the write mutex (InternalConnection.cpp line 703) and then waiting on the bolt mutex (Shared.cpp line 251). So that thread is stuck until the event handler returns. The the event handler thread attempts to publish its message and waits on the write mutex (InternalConnection.cpp line 703). So the event handler thread is stuck until the other thread finishes publishing its message. A deadlock.

Recommendation:

Ensure all mutexes are released before passing control to code you don't control.

Other option is to provide specific documentation about what actions are allowed to be taken from call backs, but that will be buggy.

The design of having several layers of mutexes seems quite complex and prone to bugs. Perhaps reducing the number of mutexes in InternalConnection and Shared could avoid these types of bugs in the future. Though I don't know what tradeoffs were made in the design.

System information:

Using the RHEL 7 build of GMSEC_API 4.6.

build error: cms/Connection.h: No such file or directory

In building I get the error below. Please help. I cannot find the file being referenced on the system. I am using the most recent zip from the repo.

In file included from ../activemq/src/gmsec_activemq.cpp:17:
../activemq/include/CMSConnection.h:27:10: fatal error: cms/Connection.h: No such file or directory
#include <cms/Connection.h>

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.