gmsec_api's Introduction
This README file provides general information regarding the GMSEC API, its supported middleware, and example programs. 1. DISTRIBUTIONS The GMSEC API is available in two different distributions (Open-Source and Government version) for the following OSes: * Windows 10 * Windows 11 * RHEL 8 * Ubuntu 20.04 LTS * Ubuntu 20.04 ARM/Aarch64 LTS * MacOS Binaries and source code for the Open-Source version is available on NASA GitHub. See here for further details: https://github.com/nasa/GMSEC_API Instructions for building the Open-Source code can be found in the INSTALL.txt file located at the root folder level where the GMSEC API project is installed. As for the Government version, it is only available in binary format (no source code), and more importantly it is only available by request from GMSEC by those supporting a US Government contract. This version contains the Compat-C2 interface (for NSS message encrption); for users that do not require this feature, then it is recommended to use the Open-Source distribution. 2. DOCUMENTATION The GMSEC API includes documentation is the docs directory which is located in the base directory where the GMSEC API has been installed. There the following documents exist: * GMSEC API Installation and Configuration Guide * GMSEC API SRS * GMSEC API User's Guide * GMSEC API VDD * GMSEC API CompatC2 Installation and Configuration Guide [see note below] * NASA C2MS Addendum [see note below] In addition, Doxygen style documentation for the GMSEC API may be found in the docs/manual folder. Open index.html for the appropriate language binding using a web browser. Note: These documents are *not* included with the Open-Source release of the GMSEC API. 3. SOFTWARE INSTALLATION and SYSTEM CONFIGURATION The GMSEC API can be installed anywhere on a system, however it is recommended to choose a folder that does not contain whitespace. Special user accounts are not necessary to use the GMSEC API, nor are admin privileges. Once the software is installed, it is recommended to reference its location using the GMSEC_API_HOME environment variable, and then to use this to configure other environment variables. For example: On Linux: export GMSEC_API_HOME=/opt/GMSEC/GMSEC_API export LD_LIBRARY_PATH=$GMSEC_API_HOME/bin:$LD_LIBRARY_PATH export PATH=$PATH:$GMSEC_API_HOME/bin export CLASSPATH=$GMSEC_API_HOME/bin/gmsecapi.jar:$CLASSPATH On Windows: set GMSEC_API_HOME=C:\GMSEC\GMSEC_API set PATH=%PATH%;%GMSEC_API_HOME%\bin set CLASSPATH=%GMSEC_API_HOME%\bin\gmsecapi.jar;%CLASSPATH% 4. MIDDLEWARE BROKERS The GMSEC API supports various middleware brokers; these include: * Apache ActiveMQ * Apache Artemis * Bolt * IBM MQ * Message Bus * OpenDDS * RabbitMQ (using AMQP 1.0) * ZeroMQ Information on acquiring, installing, and running a middleware broker may be found in the GMSEC API Installation and Configuration Guide. 5. MIDDLEWARE CLIENT LIBRARIES To interact with a middleware broker, the GMSEC API will need to rely on vendor-specific client libraries. Typically these are downloaded from the respective vendor. The exception is with Bolt and Message Bus whose libraries are included with the GMSEC API distribution. The Apache ActiveMQ client libraries (which is also used for Artemis) can be obtained by request from the GMSEC team. Alternatively, the user can build these themselves from source code. Once installed, middleware client libraries should be referenced; for example: On Linux: export LD_LIBRARY_PATH=/path/to/client/lib:$LD_LIBRARY_PATH On Windows: set PATH=%PATH%;C:\path\to\client\lib 6. UTILITIES/TOOLS The GMSEC API provides a small suite of utility/tool applications. These include: * gmconfig_edit # console app to assist creating a configuration file * gmhelp # provides general middleware information * gmlog # publishes a Log message * gmpub # publishes a Heartbeat message * gmreq # issues a Request Directive message * gmrpl # issues a Response Directive message * gmsub # subscribes to receive messages * msg_schema_ids # lists available schema IDs * throughput_pub # stress test that publishes messages * throughput_sub # stress test that subscribes to receive messages 7. SUPPORTED PROGRAMMING LANGUAGE BINDINGS The GMSEC API provides support for multiple programming languages. These include: * C * C++ * C# * Java * NodeJS * Perl * Python * Ruby Information regarding developing applications using these bindings can be found in the GMSEC API User's Guide. Notes: 1. Mono is required to use C# under Linux and MacOS 2. Strawberry Perl 5.32.1 is required use Perl under Windows 3. Python 3.10.7 (or later 3.10.x) is required to use Python 4. Ruby 3.0.2 is required to use Ruby under Windows 5. Node 18.15.0 (or later) is required for NodeJS 8. EXAMPLE PROGRAMS The GMSEC API provides a suite of example programs located under the examples directory located at the base directory where the GMSEC API distribution is installed. Each examples directory contains a README.txt file with information on how to configure one's environment to run the example programs. Those that are required to be compiled/linked will need for the working system to have development tools installed. For Linux, this will, at a minimum, require GCC and Make. For Windows, Visual Studio 2019 or later will be required. 9. SUPPORT For GMSEC API and for basic middleware support, feel free to send queries to [email protected].
gmsec_api's People
gmsec_api's Issues
activemq-cpp wrapper ignores sec-ssl-truststore unless sec-ssl-truststore-key is set
When using a CA certificate file and activemq GMSEC_API ignores the CA certificate file unless sec-ssl-truststore-key
is set to a non-empty string. See [1]. This is confusing for users since activemq-cpp does not have the capability to use a CA certificate file key. See [2].
Recommended fix is to pass along the sec-ssl-truststore
value iff it is not empty, regardless of the value of sec-ssl-truststore-key
.
[1] https://github.com/nasa/GMSEC_API/blob/master/wrapper/activemq/src/CMSConnection.cpp#L687
[2] https://github.com/apache/activemq-cpp/blob/d6f76ede90d21b7ee2f0b5d4648e440e66d63003/activemq-cpp/src/main/decaf/internal/net/ssl/openssl/OpenSSLContextSpi.cpp#L238
SIGSEGV in Message.getSubject()
As a Java developer I expect NASA's code to never segfault and kill the JVM. NullPointerExceptions are fine, segfaults are not.
Using GMSEC API version 4.6 I can produce a segfault every time by calling getSubject()
on a received REPLY
message. I can call getSubject()
on received PUBLISH
or REQUEST
messages without any problems. Reproducer code and stack trace top are below.
I assume the problem is not actually in std::string::c_str()
. My theory is that the issue is due to using reinterpret_cast<Message*>(jlong)
in the native code on some long that is not the address of a message. How NASA's Java interface got an invalid pointer is beyond me.
Current thread (0x00007f08c0012800): JavaThread "main" [_thread_in_native, id=289137, stack(0x00007f08c6ee0000,0x00007f08c6fe0000)]
siginfo: si_signo: 11 (SIGSEGV), si_code: 128 (SI_KERNEL), si_addr: 0x0000000000000000
Register to memory mapping:
RAX=0x771abb87bdb6526b is an unknown value
RBX={method} {0x00007f0878441d68} 'Message_GetSubject' '(JLgov/nasa/gsfc/gmsec/api/jni/JNIMessage;)Ljava/lang/String;' in 'gov/nasa/gsfc/gmsec/api/jni/gmsecJNI'
RCX=0x00007f08c6fdbeb0 is pointing into the stack for thread: 0x00007f08c0012800
RDX=0x00007f0828002930 is an unknown value
RSP=0x00007f08c6fdbc38 is pointing into the stack for thread: 0x00007f08c0012800
RBP=0x00007f08c6fdbc50 is pointing into the stack for thread: 0x00007f08c0012800
RSI=0x00007f08c6fdbea0 is pointing into the stack for thread: 0x00007f08c0012800
RDI=0x771abb87bdb6526b is an unknown value
R8 =0x00007f087850b928 is pointing into metadata
R9 =0x0000000000000009 is an unknown value
R10=0x00007f08b10183fb is at code_begin+859 in an Interpreter codelet
method entry point (kind = native) [0x00007f08b10180a0, 0x00007f08b1018a40] 2464 bytes
R11=0x00007f08c7a2ca00: <offset 0xa2ca00> in /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so at 0x00007f08c7000000
R12=0x0000000000000000 is an unknown value
R13={method} {0x00007f0878441d68} 'Message_GetSubject' '(JLgov/nasa/gsfc/gmsec/api/jni/JNIMessage;)Ljava/lang/String;' in 'gov/nasa/gsfc/gmsec/api/jni/gmsecJNI'
R14=0x00007f08c6fdbec0 is pointing into the stack for thread: 0x00007f08c0012800
R15=0x00007f08c0012800 is a thread
Stack: [0x00007f08c6ee0000,0x00007f08c6fe0000], sp=0x00007f08c6fdbc38, free space=1007k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C [libstdc++.so.6+0xf3c64] std::string::c_str() const+0x4
C [libGMSECAPI.so+0x314cea] gmsec::api::Message::getSubject() const+0x1c
C [libgmsec_jni.so+0x1083e3] Java_gov_nasa_gsfc_gmsec_api_jni_gmsecJNI_Message_1GetSubject+0x6b
j gov.nasa.gsfc.gmsec.api.jni.gmsecJNI.Message_GetSubject(JLgov/nasa/gsfc/gmsec/api/jni/JNIMessage;)Ljava/lang/String;+0
j gov.nasa.gsfc.gmsec.api.jni.JNIMessage.getSubject()Ljava/lang/String;+5
j gov.nasa.gsfc.gmsec.api.Message.getSubject()Ljava/lang/String;+4
public class SigsegvTest {
private static BoltServer boltServer;
@BeforeAll
public static void setup() throws InterruptedException {
// load GMSEC_API native code
GmsecLoader.loadGmsec();
// start bolt server
boltServer = new BoltServer();
new Thread(boltServer, "boltServer").start();
boltServer.waitUntilStarted();
}
@AfterAll
public static void cleanup() {
boltServer.stop();
GmsecLoader.cleanGmsec();
}
@TempDir
Path schemaDir;
@Test
public void testSegfault() throws GMSEC_Exception, IOException, InterruptedException {
// setup
Log.registerHandler(new LogHandler() {
@Override
public void onMessage(LogEntry logEntry) {
System.err.format("%s.%09d [%s] %s:%s %s\n",
logEntry.time.getSeconds(), logEntry.time.getNanoseconds(),
logEntry.level,
logEntry.fileName, logEntry.lineNumber, logEntry.message);
}
});
Log.setReportingLevel(LogLevel.VERBOSE);
Config config = new Config();
config.addValue("connectiontype", "gmsec_bolt");
config.addValue("server", "localhost");
config.addValue("mw-multi-resp", "true");
config.addValue("GMSEC-MSG-CONTENT-VALIDATE", "false");
config.addValue("GMSEC-MSG-CONTENT-VALIDATE-ALL", "false");
config.addValue("GMSEC-MSG-CONTENT-VALIDATE-SEND", "false");
config.addValue("GMSEC-MSG-CONTENT-VALIDATE-RECV", "false");
config.addValue("GMSEC-VALIDATION-LEVEL", "NO_ENFORCEMENT");
config.addValue("GMSEC-SCHEMA-LEVEL", "0");
config.addValue("GMSEC_LEGACY_SCHEMA_FILES", "true");
config.addValue("GMSEC-SPECIFICATION-VERSION", "201600");
// I wish I could turn off message validation entirely, but I can't
// so fake a directory of schemas
Files.createDirectories(schemaDir.resolve("2016.00"));
try (Writer out = Files.newBufferedWriter(schemaDir.resolve("2016.00").resolve(".DIRECTORY.xml"))) {
out.write("<DIRECTORY NAME=\"2016.00\"><SCHEMA NAME=\"HEADER\" LEVEL-0=\"GMSEC\" DEFINITION=\"\"/></DIRECTORY>");
}
try (Writer out = Files.newBufferedWriter(schemaDir.resolve("2016.00").resolve("GMSEC.HEADER.xml"))) {
out.write("<SCHEMA NAME=\"GMSEC.HEADER\"></SCHEMA>");
}
config.addValue("GMSEC-SCHEMA-PATH", schemaDir.toAbsolutePath().toString());
String subject = "GMSEC.1.2.3.4.5.6";
List<Message> messages = new ArrayList<>();
// subscribe
final ConnectionManager cm1 = new ConnectionManager(config);
cm1.initialize();
cm1.startAutoDispatch();
Thread.sleep(500);
cm1.subscribe("GMSEC.+", new ConnectionManagerCallback() {
@Override
public void onMessage(ConnectionManager connectionManager, Message message) {
// save request message
messages.add(message);
try {
if (message.getKind().equals(MessageKind.REQUEST)) {
// create and send two replies
Message reply = new Message(message.getSubject(), MessageKind.REPLY);
connectionManager.reply(message, reply);
/*
reply = new Message(message.getSubject(), MessageKind.REPLY);
connectionManager.reply(message, reply);
*/
}
} catch (GMSEC_Exception e) {
throw new RuntimeException(e);
}
}
});
// send request
try {
final ConnectionManager cm2 = new ConnectionManager(config);
cm2.initialize();
Thread.sleep(500);
Message request = new Message(subject, MessageKind.REQUEST);
cm2.request(
request,
60000,
new ConnectionManagerReplyCallback() {
@Override
public void onReply(ConnectionManager connectionManager,
Message request,
Message reply) {
// save message
messages.add(reply);
}
@Override
public void onEvent(ConnectionManager connectionManager,
Status status,
ConnectionEvent connectionEvent) {
System.err.println("" + status + " " + connectionEvent);
}
},
-1);
} catch (GMSEC_Exception | InterruptedException e) {
throw new RuntimeException(e);
}
// inelegant wait for messages to arrive
Thread.sleep(5000);
// verify saved messages
// check waited long enough for the messages to arrive
assertThat(messages.size(), is(2));
// check received request message has a subject
assertThat(messages.get(0).getSubject(), not(nullValue()));
// check received reply message has a subject
// this causes a SIGSEGV
assertThat(messages.get(1).getSubject(), not(nullValue()));
}
}
Deadlock publishing message
Came across this issue when attempting to make my application robust to the connection dropping out and reconnecting. Restarting the heartbeat thread and sending a log message that the connection was established deadlocked the application. May be related to API-5587, but I don't know the details of that issue.
Steps to reproduce:
- Create a
ConnectionManager
with an active bolt connection - Register a handler for
CONNECTION_SUCCESSFUL_EVENT
events. The handler will start a new thread that publishes a message and then publish a message of its own. - kill the bolt bus.
- restart the bolt bus
Expected result:
Step 4 triggers the handler registered in step 2 which causes two messages to be published on the bus.
Actual result:
Application deadlocks. Zero messages are published.
Analysis:
The thread dispatching the CONNECTION_SUCCESSFUL_EVENT
holds the mutex in the bolt wrapper. (Shared.cpp line 81 and 111) Control passes to the event handler while that mutex is held (Shared.cpp line 146, line 89). This means that at this point only the thread handling the event is allowed to send messages. Hopefully it doesn't try to acquire any more locks... but it will. First the thread started by handler attempts to publish a message, acquiring the write mutex (InternalConnection.cpp line 703) and then waiting on the bolt mutex (Shared.cpp line 251). So that thread is stuck until the event handler returns. The the event handler thread attempts to publish its message and waits on the write mutex (InternalConnection.cpp line 703). So the event handler thread is stuck until the other thread finishes publishing its message. A deadlock.
Recommendation:
Ensure all mutexes are released before passing control to code you don't control.
Other option is to provide specific documentation about what actions are allowed to be taken from call backs, but that will be buggy.
The design of having several layers of mutexes seems quite complex and prone to bugs. Perhaps reducing the number of mutexes in InternalConnection
and Shared
could avoid these types of bugs in the future. Though I don't know what tradeoffs were made in the design.
System information:
Using the RHEL 7 build of GMSEC_API 4.6.
build error: cms/Connection.h: No such file or directory
In building I get the error below. Please help. I cannot find the file being referenced on the system. I am using the most recent zip from the repo.
In file included from ../activemq/src/gmsec_activemq.cpp:17:
../activemq/include/CMSConnection.h:27:10: fatal error: cms/Connection.h: No such file or directory
#include <cms/Connection.h>
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.