Giter VIP home page Giter VIP logo

das2c's Introduction

das2C - version 3.0 (in work)

This version will support parsing das3 streams without namespaces, das3 docs will come latter on a subsequent release

Das2 servers typically provide data relavent to space plasma and magnetospheric physics research. To retrieve data, an HTTP GET request is posted to a das2 server by a client program and a self-describing stream of data values covering the requested time range, at the requested time resolution, is provided in the response body.

This package, das2C, provides a portable C library, libdasC, which contains functions for:

  • Stream reading and writing
  • Streaming power spectral density estimation
  • General SI unit manipulation
  • Dataset accumulation
  • Federated catalog navigation

as well as a set of command line das2 stream processing programs used by dasflex.

Doxygen library documentation is available in the github pages for this repository. To find out more about das2 visit https://das2.org.

Packages: Anaconda, RPM, etc.

Anaconda Package

The easiest way to get das2C is just to use a package manager. The one with the most support is Anaconda, as das2C is required for das2py. To install in miniconda/anaconda:

source miniconda3/bin/activate
conda install -c dasdevelopers das2c

For Red Hat/CentOS/Rocky users you can:

curl -OJ https://github.com/das-developers/das2C/releases/download/v2.3.0/das2C-2.3.0-1.el8.x86_64.rpm
curl -OJ https://github.com/das-developers/das2C/releases/download/v2.3.0/das2C-devel-2.3.0-1.el8.x86_64.rpm
dnf localinstall das2C*.rpm

Building the quick way: XMake

Two build systems are provided for das2C. Plain ole GNU Make and Microsoft NMake files, and an xmake file. Since xmake is both a package manager and a build tool, you do not need to install any prerequisites to build das2C with xmake, other then your compiler and xmake itself.

To build and install das2C via xmake issue:

xmake config -m debug  # or 'release' if you prefer
xmake build
xmake run unittest
xmake install          # For a POSIX style /usr/local install
#or
xmake install -o C:\local   # A windows equivalent

If for some reason, xmake doesn't work for you, read on for manual dependency download and build instructions.

Manual build prequisites

Compliation of installation of das2c has been tested on Linux, Windows, MacOS, and Android. The following common system libraries are required to build libdas2.3:

  • expat - XML Parsing library
  • fftw3 - Fastest Fourier Transform in the West, version 3.
  • openssl - Secure network socket library

Though package names vary from system to system, commands for installing the prequisites are provided below ...

$ sudo yum install expat-devel fftw-devel openssl-devel # CentOS 7 and similar
$ sudo yum install python3-devel python36-numpy         # CentOS 7 and similar

$ sudo apt install libexpat-dev libfftw3-dev libssl-dev zlib1g-dev    # Debian 9 and similar

and on windows using vcpkg...

> vcpkg install openssl fftw3 zlib expat pthreads --triplet x64-windows-static

or on mac using brew

$ brew install openssl
$ brew install fftw

The expat library should already be present on MacOS once the compiler install command xcode-select --install has been run.

Manual Build and Install

Decide where you want to install the software. In the instructions below we've chosen /usr/local but anywhere is fine.

For POSIX compliant systems (Linux, MacOS, Android) issue the following commands to build, test and install the software.

$ export PREFIX=/usr/local
$ export N_ARCH=/               # For generic builds, omit per-OS sub-directories.
$ make
$ make test
$ make install

For Windows systems issue the following commands in a command shell to build, test and install the software.

> set N_ARCH=\
> set LIBRARY_INC=      ::location of your vcpkg installed\x64-windows-static include 
> set LIBRARY_LIB=      ::location of your vcpkg installed\x64-windows-static lib
> set INSTALL_PREFIX=C:\opt   :: for example
> nmake.exe /nologo /f buildfiles\Windows.mak build
> nmake.exe /nologo /f buildfiles\Windows.mak run_test
> nmake.exe /nologo /f buildfiles\Windows.mak install

Usage

By default all header files are copied into the subdirectory das2 under $PREFIX/include. When writing code that uses das2 headers add the include directory to the compiler command line in a manner similar to the following:

-I $PREFIX/include 
/I %PREFIX%/include

and use the das2 subdirectory in your include statements, for example:

#include <das2/core.h>

Common linker arguments for building libdas2.3 dependent applications follow. For open source programs static linking is perfectly fine:

$(PREFIX)/lib/libdas2.3.a -lexpat -lssl -lcrypto -lz -lm -lpthread                      # gnu make

$(INSTALL_PREFIX)/lib/libdas2.3.lib  Advapi32.lib User32.lib Crypt32.lib ws2_32.lib     # win nmake

For closed source applications, link against shared das2 objects (i.e. libdas2.3.so or das2.3.dll) as required by the LGPL:

-L$(PREFIX)/lib -ldas2.3 -lexpat -lssl -lcrypto -lz -lm -lpthread                       # gnu make

/L $(INSTALL_PREFIX)\bin das2.3.dll das2.3.lib Advapi32.lib User32.lib Crypt32.lib ws2_32.lib  # win nmake

Note that on Windows, libdas2.3.lib is the full static library but the file das2.3.lib is merely a DLL import library.

Reporting bugs

Please use the issue tracker for the das2C github.com project to report any problems with the library. If you've fixed a bug,

  1. thanks!, 2) please send a pull request so that your updates can be pulled into the main project.

Appreciation

Though most of the code in das2C was written by das developers, the vast majority of the code in the files json.h and json.c in the das2 subdirectory is from Neil Henning's elegant and efficient JSON parser, json.h. Thanks Neil for sharing your parser with the rest of the world, it's an example of classic C programming at it's best.

das2c's People

Contributors

cpiker avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

das2c's Issues

parsetime is locale aware

The parsetime routine in time.c fails when the current locale uses a comma , for the decimal separator but a time is given using a period . for the sub-seconds separator. This breaks unit tests in das2D on windows. The suggested fix to to allow both period and comma since any given data file may originate from users in various locales.

make frames manditory

The vecClass attribute of element <vector> is not parsed in the serialization module, for good reason. The <frame> element defines the vector class.

Right now the stream XSD schema allows for undefined frames to be created by adhoc by vector variables without a formal frame definition. When they do it's split between the dimension level and variable level which can lead to inconsistent results.

If frames definition were mandatory, vecClass would not be needed and frame names could be moved to the variable elements.

Console IO for password protected sources fails under Windows+IDL

The default das_term_prompt function in credentials.c is expecting ASCII input for username and password data, however the console could be set to use any random code page.

In addition, the das_b64_encode function (also in credentials.c) seems to produce illegal base64 strings for windows console input.

To replicate the problem:

  1. Start IDL
  2. Do not call das2c_credload()
  3. In order to trigger console IO, access a password protected data source.
  4. View the resulting hash via das2c_creds(), it will not be a legal base 64 encoding.

Though the bug replication procedure above uses IDL, and thus the Das2DLM, the cause seems to be in das2C code.

Buffer output size determination for DasDs_toStr()

Currently, there is no way to know how many bytes are needed to retrieve the string representation of a dataset returned
by DasDs_toStr(). The write-up for this function just says to provide a good sized buffer. Would it be possible to modify
this routine in order to return the number of characters needed to print the description so that the appropriate amount of
space could be allocated? Or perhaps another routine created to do precisely this function?

Remove units from DasAry

In addition to it's job of being flexibly indexed compact value storage, DasAry carries units. However we have a structure for carrying units plus a range of values and that is a DasVar.

Units should be removed from DasAry. It is an unnecessary complication as the units invariably are assigned UNIT_DIMENSIONLESS in most cases. In the instances where they are not, it's in association with a DasVar mapping.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.