CRSP-Engine
Continuous RDF Stream Processing Engine (CRSP Engine) using RSPQL
Our Continuous RSPQL Stream Processing (CRSP) Engine is a new tool that allows for continuous RSP-QL queries to be applied over streams of RDF graph data. The primary goal for our system is to be able to run any valid RSP-QL query over any valid RDF graph stream and output the correct result.
Installation Dependancy Prerequisites:
- Java version: 1.8 - below we will show how to install java 1.8 jdk on Linux.
Installing Java JDK 1.8 on Linux
Create java folder in opt and make it your working directory.
sudo mkdir /opt/java && cd /opt/java
Download java 1.8 jdk
.tar file.
Copy downloaded file into /opt/java
.
sudo mv ~/Downloads/jdk-8u161-linux-x64.tar.gz /opt/java
Unzip jdk, then remove original zip.
sudo tar -zxvf jdk-8u161-linux-x64.tar.gz
sudo rm -rf jdk-8u161-linux-x64.tar.gz
Update your alternatives entry for java
, javac
, jar
telling the system where they are installed.
sudo update-alternatives --install /usr/bin/java java /opt/java/jdk1.8.0_161/bin/java 100
sudo update-alternatives --config java
sudo update-alternatives --install /usr/bin/javac javac /opt/java/jdk1.8.0_161/bin/javac 100
sudo update-alternatives --config javac
sudo update-alternatives --install /usr/bin/jar jar /opt/java/jdk1.8.0_161/bin/jar 100
sudo update-alternatives --config jar
Configure environment variables:
export JAVA_HOME=/opt/java/jdk1.8.0_161/
export JRE_HOME=/opt/java/jdk1.8.0_161/jre
export PATH=$PATH:/opt/java/jdk1.8.0_161/bin:/opt/java/jdk1.8.0_161/jre/bin
Confirm Java setup:
java -version
# will output:
java version "1.8.0_161"
Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)
Installation
OSX & Linux & Windows:
git clone https://github.com/FraserBrown/CRSP-Engine.git .
Running/Using CRSP Engine:
The CRSP Engine comes with a GUI to make interaction with our system easier.
- To run the GUI please enter the following from the CRSP-Engine folder:
java -jar out/artifacts/crsp_engine_jar/crsp-engine.jar
You should see the following:
- Please specify a .json file that contains the json structure you want. The json file should be stored in your project folder.
- For the query input you have 2 options :
- specify the text file containing the query. The text file should be stored in your project folder (same folder as your json file)
- manually input the query in the "Query String" box
-
Specify the name for the output file where you wish your results to be saved. The file itself will be created and saved in the project folder (same folder as the json file).
-
Once you have setup the desired input, press the "Process Query" button. This will process the received input, create the output file and store the output in it as well as printing the output in the "Query Results" box.
Contributing
Git Branch Naming Convention
Depending on the type of contribution you are making please name your branch with one of the following tags in this format (TAG/your_feature_name)
- feature
- hotfix
- bugfix
Git Project Workflow
This project uses a git-branch based workflow, we have 3 main branches
master
- This holds stable code that are treated as releases.dev
- This holds current development build, all development bugs, features and hotfixes are branched fromdev
gh-pages
- Source code for our github pages website
Setting Up Git
- Clone our code, and checkout our dev branch
$ git clone https://github.com/FraserBrown/CRSP-Engine.git
$ git checkout dev
- Create new
branch
from thedev
branch
$ git checkout -b your_tag/your_branch_name dev
- Develop your code, commit code, and push changes to your new branch.
NOTE: Your first push of your to your branch with new changes will have to be in the following format:
git push -u origin your_tag/your_branch_name
- When development is complete create a pull request.
Development Environment
Packages used in this project
The below packages will be installed through maven automatically for you:
- RDF4J - Framework for processing and handling RDF data
- GSON v2.3.1 - JSON to Java Object Parser
- JUnit v4.12 - Testing Framework
- TravisCi - Continuous Integration software used in the git repository.
Set up Eclipse
Pre-Requisites:
- Install JDK1.8 as shown in Installing Java JDK 1.8 on Linux.
Clone
the repository and set you your newbranch
as per contributers setting up git.- Install Eclipse
Import Poject into Eclipse:
-
Open Eclipse
and enter thepath
to the parent diretory of the previously cloned repository. -
Once eclipse has started select
File --> Import
-
Select
maven --> Existing Maven Projects
then clicknext
-
Browse to the newly created project, select the pom file from box, click
Finish
The project should import dependencies will be downloaded through the pom file and maven in the background. You will can now start developing.
Testing Environment
The testing environment in this project is done through JUnit, tests can be found in: CRSP-Engine/src/test/java/
. Our test naming scheme uses the following format; *Test.java
where *
is the name of the feature or system element you wish to test.
Some examples include:
- RSPQLParseTest.java
To run all the test in to perform regression testing (testing your new feature has not broken any existing functionality):
- Open Eclipse or your IDE of choice
- In the
project pane
(on the left hand side) right click on the directorytest/java
then selectRun All Tests
.
Continious Integration
Out git repository uses continiuous integration in the form of TravisCI. This runs a clean build and any tests in the src/test/java/
directory when ever a new change is pushed to a branch.