Giter VIP home page Giter VIP logo

database's Introduction

Welcome to the Blazegraph Database

Blazegraph™ DB is a ultra high-performance graph database supporting Blueprints and RDF/SPARQL APIs. It supports up to 50 Billion edges on a single machine. It is in production use for Fortune 500 customers such as EMC, Autodesk, and many others. It is supporting key Precision Medicine applications and has wide-spread usage for life science applications. It is used extensively to support Cyber analytics in commercial and government applications. It powers the Wikimedia Foundation's Wikidata Query Service.

image

Please see the release notes in releases for version changes.

Sign up to get the latest news on Blazegraph.

Please also visit us at our: website, wiki, and blog.

Find an issue? Need help? See JIRA.

Reporting a security issue: Security Reporting.

Quick Start with the Executable Jar

Up and running with Blazegraph in under 30 seconds: Quick Start.

Deploying in Production

Blazegraph is designed to be easy to use and get started. It ships without SSL or authentication by default for this reason. For production deployments, we strongly recommend you enable SSL, authentication, and appropriate network configurations. There are some helpful links below to enable you to do this.

Enabling SSL support

To enable SSL support, uncomment the example jetty.xml and configure it for your local keystore.

Configuration Authentication

By default, Blazegraph ships without authentication enabled. This is great for developing, getting started, and doing research with Blazegraph. However, it's not recommended for any production deployment. To configuration authentication, you must configure it either within the web app container or via a reverse-proxy configuration.

Note that the Blazegraph namespace feature for multi-tenancy does not provide security isolation. Users that can access the base URI of the server can access any of the available namespaces. You can further restrict this through a combination of authentication configuration and restricting access to specific namespace URIs, i.e. /blazegraph/namespace/NAMESPACE/sparql.

There are three basic options:

  1. Configuring Jetty Authentication for a standalone Jetty deployment: Follow the jetty guide to configure authentication for the jetty.xml you use to deploy the server by uncommenting the <Get name="securityHandler"> section. You'll need to create a realm.properties and update the jetty.xml to point to its location on the filesystem. Then configure the web.xml to uncomment the security-constraint.
  2. Configuring Tomcat Authentication for a standalone Tomcat deployment: First configure a Tomcat Realm with your choice of authentication method (JDBC, JNDI, etc.). Then configure the web.xml to uncomment the security-constraint.
  3. Setup a reverse-proxy configuration with authentication: You can setup an http or https reverse proxy configuration that has authentication and forward requests to the local Blazegraph instance (typically running on localhost:9999). This is a good option with Nginx and Apache.
Mitigating Cross-Site Request Forgery (CSRF)

If you enable authentication and expose the Blazegraph workbench, you should also take steps to protect against CSRF. Tomcat8 provides a CSRF filter that can be configured. For Jetty, if you configure authentication the default value for SecurityHandler.setSessionRenewedOnAuthentication(true) can also be used. CSRF protection may require REST clients to implement HTTP headers to be used to interact with the service.

Building the code

As a quick start, run mvn install -DskipTests or the utility script ./scripts/mavenInstall.sh .

For more detailed maven information see the wiki.

If you build with Java 7, you need to add Maven options for TLS 1.2, i.e. export MAVEN_OPTS="-Dhttps.protocols=TLSv1.2".

Samples and Examples

There are code samples and examples to get started with the Blazegraph Database [here] (https://github.com/blazegraph/blazegraph-samples). Tinkerpop3 examples are included directly within the Tinkerpop3 repository per below.

Javadocs

Click here to view the lastest API Javadocs.

Maven Central

Starting with the 2.0.0 release, the Blazegraph Database is available on Maven Central. To include the core platform and dependencies, include the artifact below in your dependencies. Developing with Maven has notes on developing with Blazegraph Database source code and Maven.

    <dependency>
        <groupId>com.blazegraph</groupId>
        <artifactId>bigdata-core</artifactId>
        <version>2.0.0</version>
    </dependency>
    <!-- Use if Tinkerpop 2.5 support is needed ; See also Tinkerpop3 below. -->
    <dependency>
        <groupId>com.blazegraph</groupId>
        <artifactId>bigdata-blueprints</artifactId>
        <version>2.0.0</version>
    </dependency>

If you'd just link the Blazegraph Database dependencies without any of the external libraries, use the bigdata-runtime artifact.

    <dependency>
        <groupId>com.blazegraph</groupId>
        <artifactId>bigdata-runtime</artifactId>
        <version>2.0.0</version>
    </dependency>

Deployers

Starting with 2.0.0, the default context path for deployment is http://localhost:9999/blazegraph/. There are also Maven artifacts for WAR deployers (blazegraph-war), executable Jar files (blazegraph-jar), Debian Package (blazegraph-deb), RPM (blazegraph-rpm), and a Tarball (blazegraph-tgz).

The bigdata-war and bigdata-jar artifacts are included for legacy purposes and use the /bigdata/ context path.

Tinkerpop3

Tinkerpop3 supports requires Java 1.8 and is now in a separate repository. See Tinkerpop3. It is also available as Maven Central artifact.

    <dependency>
        <groupId>com.blazegraph</groupId>
        <artifactId>blazegraph-gremlin</artifactId>
        <version>1.0.0</version>
    </dependency>
    

Triple Pattern Fragment (TPF) Server

There is a Blazegraph Triple Pattern Fragment TPF server that supports Linked Data Fragments.

Blazegraph Python Client

There is a Blazegraph Python Client here

Blazegraph Dot Net RDF Client

There is a Blazegraph Dot Net RDF Client here

database's People

Contributors

beebs-systap avatar binrush avatar cvedetect avatar hartig avatar hashar avatar igor-kim avatar jlleitschuh avatar mikepersonick avatar mschmidt00 avatar phaasenase avatar smalyshev avatar thompsonbry avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

database's Issues

Can not build release BLAZEGRAPH_RELEASE_2_1_2 successfully

I git clone the release BLAZEGRAPH_RELEASE_2_1_2, and build it as document, but it can't run successfully. Commands are as follow:
git clone -b BLAZEGRAPH_RELEASE_2_1_2 https://github.com/blazegraph/database.git BLAZEGRAPH_RELEASE_2_1_2
ant deploy-artifact
The error messages are as follow:
`jar:

BUILD FAILED
/root/blazegraph/git/BLAZEGRAPH_RELEASE_2_1_2/build.xml:250: /root/blazegraph/git/BLAZEGRAPH_RELEASE_2_1_2/ant-build/classes not found.`
What should I do with this?

Replicating BlazeGraph DB

Hi,

I'm having blazegraph database which is out of my control, and I'd like to set-up clone (duplicate, slave).
Is there any way you could recommend? Only access to the remote DB is through SPARQL HTTP interface.

Thank you!

TimeoutException on Raspberry Pi

Hi,

I'm trying to start blazegraph on a Raspberry Pi 2 B+ with: java -server -Xmx800m -jar blazegraph.jar.

The error I'm getting is:

WARN : NanoSparqlServer.java:514: Starting NSS
WARN : ServiceProviderHook.java:171: Running.
Server did not start.
FATAL: NanoSparqlServer.java:535: Server did not start.
ERROR: Banner.java:160: Uncaught exception in thread
java.util.concurrent.TimeoutException
    at com.bigdata.rdf.sail.webapp.NanoSparqlServer.awaitServerStart(NanoSparqlServer.java:525)
    at com.bigdata.rdf.sail.webapp.StandaloneNanoSparqlServer.main(StandaloneNanoSparqlServer.java:150)

Is this due to a low resource environment? Is there anyway to set the start server timeout longer?
Cheers!

question

i d like to try blazegraph . for my opinion is the best graph database. I m a curiosity about query language:
blazegraph makes a index call (on internal indexer) for every vertex subquery and edge subquery present in the main query? Pratically blazegraph make many index calls for every query right?

Blazegraph java API

Hi, I'am iterating a large resultset (5M of URIs) with blazegraph java api (openrdf based):
`
try {
final TupleQueryResult results = repository
.prepareTupleQuery(config.getSelectQuery())
.evaluate();

final Collection<String> uriGroup = new ArrayList<>();

while(results.hasNext()) {
    final BindingSet result = results.next();
    final String uri = result.getBinding(variable).getValue().stringValue();
    uriGroup.add(uri);

    if(uriGroup.size() == batchSize) {
        makeConstruct(uriGroup, config, repository);
        uriGroup.clear();
    }
}

if(uriGroup.size() > 0) {
    makeConstruct(uriGroup, config, repository);
    uriGroup.clear();
}

} catch (final Exception e) {
e.printStackTrace();
}
`

For each group of uris (uriGroup) I perform another query on the server:
`
private void makeConstruct(final Collection uris, final SparqlPumpConfig config, final RemoteRepository repository) {

final String query = createConstruct(config, uris);

try {
    final GraphQueryResult graph = repository
        .prepareGraphQuery(query)
        .evaluate();

    while(graph.hasNext()) {
        final Statement statement = graph.next();

        final String subjectUri = statement.getSubject().stringValue();
        final String propertyUri = statement.getPredicate().stringValue();
        final Object object = cast(statement.getObject());

        // ... do something ...

    }

            graph.close();

} catch (final Exception e) {
    log.info("error executing query: {}", query);
    e.printStackTrace();
}

}
`

Unfortunately in the middle of the execution the client stops reporting the following exception:
org.openrdf.query.QueryEvaluationException: java.io.IOException: Unkown record type: 10 at org.openrdf.http.client.QueueCursor.checkException(QueueCursor.java:164) at org.openrdf.http.client.QueueCursor.getNextElement(QueueCursor.java:123) at info.aduna.iteration.LookAheadIteration.lookAhead(LookAheadIteration.java:91) at info.aduna.iteration.LookAheadIteration.hasNext(LookAheadIteration.java:60) at info.aduna.iteration.IterationWrapper.hasNext(IterationWrapper.java:68) at info.aduna.iteration.IterationWrapper.hasNext(IterationWrapper.java:68) at info.aduna.iteration.IterationWrapper.hasNext(IterationWrapper.java:68) at com.bigdata.rdf.sail.webapp.client.RemoteRepositoryManager$2.hasNext(RemoteRepositoryManager.java:2227)
[...]
Caused by: java.io.IOException: Unkown record type: 10 at org.openrdf.query.resultio.binary.BinaryQueryResultParser.parse(BinaryQueryResultParser.java:197) at com.bigdata.rdf.sail.webapp.client.BackgroundTupleResult.run(BackgroundTupleResult.java:126) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

Do you think am I using the API in the wrong way or maybe there's some bug?

Thanks,
Riccardo

SPARQL CONTAINS vs bds:search performance

When using a SPARQL FILTER CONTAINS in a WHERE clause it seems to take twice as long to get the same results as using a bds:search predicate instead of the CONTAINS. This behavior was observed even with the Runtime Query Optimizer enabled. Therefore, I suspect there may be an opportunity to have the Blazegraph implementation automatically make use of the full text search features (https://wiki.blazegraph.com/wiki/index.php/FullTextSearch) when a CONTAINS condition is present in a SPARQL query and the full text index is present. It seems that the current implementation of CONTAINS is independent of the full text index feature.

The Application can't stop

I run the server with command :java -server -Xmx4g -jar blazegraph.jar。
then I run a Application in Eclipse,code is below:
public static void main(String[] args) throws Exception {
protected static final Logger log = Logger.getLogger(Test.class);
final BigdataGraph graph = new BigdataGraphClient("http://localhost:9999/blazegraph/sparql");
graph.addVertex(null);
graph.shutdown();
log.info("end");
}
this Application can't stop,the console already output the line: end, Why?

maven fails to build blazegraph-rpm artifact

Valid for BLAZEGRAPH_2_1_4, 2_1_2 and maybe others. In whichever way I try to build the "rpm artifact" I get a fatal error.

$ rpm -q rpm-build
rpm-build-4.8.0-37.el6.x86_64

$ java -version
openjdk version "1.8.0_101"
OpenJDK Runtime Environment (build 1.8.0_101-b13)
OpenJDK 64-Bit Server VM (build 25.101-b13, mixed mode)

The error presented (below) gives me now clue as to how to remedy this.
Here is the summary output of the build process:

[INFO] ------------------------------------------------------------------------
[INFO] Building Blazegraph RPM Package 2.1.4
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ blazegraph-rpm ---
[INFO]
[INFO] --- rpm-maven-plugin:2.1.5:version (default-version) @ blazegraph-rpm ---
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Blazegraph Deployment Artifacts .................... SUCCESS [  0.150 s]
[INFO] Blazegraph WAR (bigdata legacy context path) ....... SUCCESS [  3.534 s]
[INFO] Blazegraph Executable Jar (bigdata legacy path) .... SUCCESS [ 11.331 s]
[INFO] Blazegraph WAR ..................................... SUCCESS [  2.246 s]
[INFO] Blazegraph Executable Jar .......................... SUCCESS [  9.394 s]
[INFO] Blazegraph Debian Package .......................... SUCCESS [  7.610 s]
[INFO] Blazegraph RPM Package ............................. FAILURE [  0.046 s]
[INFO] Blazegraph Tarball Package ......................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 35.909 s
[INFO] Finished at: 2016-09-09T10:24:19+02:00
[INFO] Final Memory: 33M/195M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:rpm-maven-plugin:2.1.5:version (default-version) on project blazegraph-rpm: Execution default-version of goal org.codehaus.mojo:rpm-maven-plugin:2.1.5:version failed: Unable to load the mojo 'version' (or one of its required components) from the plugin 'org.codehaus.mojo:rpm-maven-plugin:2.1.5': java.util.NoSuchElementException
[ERROR] role: org.apache.maven.plugin.Mojo
[ERROR] roleHint: org.codehaus.mojo:rpm-maven-plugin:2.1.5:version

Nullpointer on SPARQL-UPDATE DELETE

Dear Blazegraph community

thank you for sharing blazegraph. I am still trying to use a journal with truth maintenance, because I still believe in it (inside blazegraph, as SYSTAP tells). Sometime working with blagzegraph gives indeed some headaches ...

What I did:

I loaded a couple of OWL ontologies (this time USING sesame's file load function...) and the RDF loaded pretty smoothly. I need those ontologies - they are taken from the tree of FIBO (Subtree (=all imported ontologies) from http://www.omg.org/spec/EDMC-FIBO/FND/20141101/Law/Jurisdiction.rdf onwards).

The OWL stuff is loaded into a journal namespace with truth maintenance. In this namespace there are, besides the loaded OWL stuff, some further RDF plus the following three triples connected with subject http://semweb.ch/lodowl/0:

select * { <http://semweb.ch/lodowl/0> ?p ?o. }

gives:

po
rdf:typeskos:ConceptScheme
rdfs:labelInsert the name of a new Concept Scheme
skos:prefLabelInsert the name of a new Concept Scheme

And now the "pleasure": The following needed query is put in the SPARQL-UPDATE window of the workbench but is also obtained sending via http to blazegraph using HTTPRepository()

prefix skos: <http://www.w3.org/2004/02/skos/core#> DELETE {<http://semweb.ch/lodowl/0> skos:prefLabel ?x } WHERE {<http://semweb.ch/lodowl/0> skos:prefLabel ?x}

After that query is submitted as SPARQL-UPDATE, a HUGE exception is sent back telling that there is "somewhere" a null pointer ... (see below).

The repository properties are:

demo_TESTOWL

com.bigdata.rdf.store.AbstractTripleStore.justifytrue;
com.bigdata.rdf.sail.namespacedemo_TESTOWL
com.bigdata.relation.classcom.bigdata.rdf.store.LocalTripleStore
com.bigdata.rdf.sail.truthMaintenancetrue
com.bigdata.btree.writeRetentionQueue.capacity16000
com.bigdata.rdf.store.AbstractTripleStore.textIndextrue
com.bigdata.search.FullTextIndex.fieldsEnabledfalse
com.bigdata.relation.namespacedemo_TESTOWL
com.bigdata.rdf.store.AbstractTripleStore.vocabularyClasscom.bigdata.rdf.vocab.core.BigdataCoreVocabulary_v20151210
com.bigdata.rdf.store.AbstractTripleStore.quadsfalse
com.bigdata.relation.containerdemo_TESTOWL
com.bigdata.journal.AbstractJournal.bufferModeDiskRW
com.bigdata.rdf.sail.bufferCapacity5000000

Status of Blazegraph

Build Version=2.0.0

Build Git Commit=516e5a7

Build Git Branch=${git.branch}

Accepted query count=32

Running query count=0

Show queries, query details.

/GeoSpatial/bigMinCalculationTimeMS=0 /GeoSpatial/filterCalculationTimeMS=0 /GeoSpatial/geoSpatialSearchRequests=0 /GeoSpatial/geoSpatialServiceCallSubRangeTasks=0 /GeoSpatial/geoSpatialServiceCallTasks=0 /GeoSpatial/rangeCheckCalculationTimeMS=0 /GeoSpatial/zOrderIndexHitRatio=null /GeoSpatial/zOrderIndexHits=0 /GeoSpatial/zOrderIndexMisses=0 /GeoSpatial/zOrderIndexScannedValues=0 /blockedWorkQueueCount=0 /blockedWorkQueueRunningTotal=0 /deadlineQueueSize=0 /operatorActiveCount=0 /operatorHaltCount=421 /operatorStartCount=421 /operatorTasksPerQuery=13.15625 /queriesPerSecond=146.78899082568807 /queryDoneCount=32 /queryErrorCount=0 /queryStartCount=32

What can I do to avoid that exception, what should be done? What can be wrong?

Thanks
Fabio

Here the exception with Blazegraph 2.0.0:
ERROR: SPARQL-UPDATE: updateStr=prefix skos: prefix skosxl: prefix dcterms: prefix semwebo: DELETE { skos:prefLabel ?x } WHERE { skos:prefLabel ?x} java.util.concurrent.ExecutionException: java.util.concurrent.ExecutionException: org.openrdf.query.UpdateExecutionException: java.lang.NullPointerException at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at com.bigdata.rdf.sail.webapp.BigdataServlet.submitApiTask(BigdataServlet.java:281) at com.bigdata.rdf.sail.webapp.QueryServlet.doSparqlUpdate(QueryServlet.java:448) at com.bigdata.rdf.sail.webapp.QueryServlet.doPost(QueryServlet.java:229) at com.bigdata.rdf.sail.webapp.RESTServlet.doPost(RESTServlet.java:269) at com.bigdata.rdf.sail.webapp.MultiTenancyServlet.doPost(MultiTenancyServlet.java:170) at javax.servlet.http.HttpServlet.service(HttpServlet.java:648) at javax.servlet.http.HttpServlet.service(HttpServlet.java:729) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:230) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:108) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:522) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:1096) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:760) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1480) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745) Caused by: java.util.concurrent.ExecutionException: org.openrdf.query.UpdateExecutionException: java.lang.NullPointerException at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at com.bigdata.rdf.sail.webapp.QueryServlet$SparqlUpdateTask.call(QueryServlet.java:552) at com.bigdata.rdf.sail.webapp.QueryServlet$SparqlUpdateTask.call(QueryServlet.java:460) at com.bigdata.rdf.task.ApiTaskForIndexManager.call(ApiTaskForIndexManager.java:68) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ... 1 more Caused by: org.openrdf.query.UpdateExecutionException: java.lang.NullPointerException at com.bigdata.rdf.sparql.ast.eval.ASTEvalHelper.executeUpdate(ASTEvalHelper.java:1080) at com.bigdata.rdf.sail.BigdataSailUpdate.execute2(BigdataSailUpdate.java:152) at com.bigdata.rdf.sail.webapp.BigdataRDFContext$UpdateTask.doQuery(BigdataRDFContext.java:1966) at com.bigdata.rdf.sail.webapp.BigdataRDFContext$AbstractQueryTask.innerCall(BigdataRDFContext.java:1568) at com.bigdata.rdf.sail.webapp.BigdataRDFContext$AbstractQueryTask.call(BigdataRDFContext.java:1533) at com.bigdata.rdf.sail.webapp.BigdataRDFContext$AbstractQueryTask.call(BigdataRDFContext.java:705) ... 4 more Caused by: java.lang.NullPointerException at com.bigdata.rdf.inf.FullyBufferedJustificationIterator.(FullyBufferedJustificationIterator.java:98) at com.bigdata.rdf.inf.Justification.isGrounded(Justification.java:809) at com.bigdata.rdf.inf.Justification.isGrounded(Justification.java:641) at com.bigdata.rdf.inf.TruthMaintenance.retractAll(TruthMaintenance.java:850) at com.bigdata.rdf.inf.TruthMaintenance.retractAll(TruthMaintenance.java:612) at com.bigdata.rdf.sail.BigdataSail$BigdataSailConnection.flushStatementBuffers(BigdataSail.java:3934) at com.bigdata.rdf.sail.BigdataSail$BigdataSailConnection.commit2(BigdataSail.java:3687) at com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit2(BigdataSailRepositoryConnection.java:330) at com.bigdata.rdf.sparql.ast.eval.AST2BOpUpdate.convertCommit(AST2BOpUpdate.java:375) at com.bigdata.rdf.sparql.ast.eval.AST2BOpUpdate.convertUpdate(AST2BOpUpdate.java:321) at com.bigdata.rdf.sparql.ast.eval.ASTEvalHelper.executeUpdate(ASTEvalHelper.java:1072) ... 9 more

And here the exception using Blazegraph 2.1.4:
ERROR: SPARQL-UPDATE: updateStr=prefix skos: DELETE { skos:prefLabel ?x } WHERE { skos:prefLabel ?x} java.util.concurrent.ExecutionException: java.util.concurrent.ExecutionException: org.openrdf.query.UpdateExecutionException: java.lang.NullPointerException at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at com.bigdata.rdf.sail.webapp.BigdataServlet.submitApiTask(BigdataServlet.java:281) at com.bigdata.rdf.sail.webapp.QueryServlet.doSparqlUpdate(QueryServlet.java:458) at com.bigdata.rdf.sail.webapp.QueryServlet.doPost(QueryServlet.java:239) at com.bigdata.rdf.sail.webapp.RESTServlet.doPost(RESTServlet.java:269) at com.bigdata.rdf.sail.webapp.MultiTenancyServlet.doPost(MultiTenancyServlet.java:193) at javax.servlet.http.HttpServlet.service(HttpServlet.java:648) at javax.servlet.http.HttpServlet.service(HttpServlet.java:729) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:230) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:108) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:522) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:1096) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:760) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1480) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745) Caused by: java.util.concurrent.ExecutionException: org.openrdf.query.UpdateExecutionException: java.lang.NullPointerException at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at com.bigdata.rdf.sail.webapp.QueryServlet$SparqlUpdateTask.call(QueryServlet.java:569) at com.bigdata.rdf.sail.webapp.QueryServlet$SparqlUpdateTask.call(QueryServlet.java:470) at com.bigdata.rdf.task.ApiTaskForIndexManager.call(ApiTaskForIndexManager.java:68) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ... 1 more Caused by: org.openrdf.query.UpdateExecutionException: java.lang.NullPointerException at com.bigdata.rdf.sparql.ast.eval.ASTEvalHelper.executeUpdate(ASTEvalHelper.java:1080) at com.bigdata.rdf.sail.BigdataSailUpdate.execute2(BigdataSailUpdate.java:152) at com.bigdata.rdf.sail.webapp.BigdataRDFContext$UpdateTask.doQuery(BigdataRDFContext.java:1968) at com.bigdata.rdf.sail.webapp.BigdataRDFContext$AbstractQueryTask.innerCall(BigdataRDFContext.java:1569) at com.bigdata.rdf.sail.webapp.BigdataRDFContext$AbstractQueryTask.call(BigdataRDFContext.java:1534) at com.bigdata.rdf.sail.webapp.BigdataRDFContext$AbstractQueryTask.call(BigdataRDFContext.java:747) ... 4 more Caused by: java.lang.NullPointerException at com.bigdata.rdf.inf.FullyBufferedJustificationIterator.(FullyBufferedJustificationIterator.java:98) at com.bigdata.rdf.inf.Justification.isGrounded(Justification.java:809) at com.bigdata.rdf.inf.Justification.isGrounded(Justification.java:641) at com.bigdata.rdf.inf.TruthMaintenance.retractAll(TruthMaintenance.java:850) at com.bigdata.rdf.inf.TruthMaintenance.retractAll(TruthMaintenance.java:612) at com.bigdata.rdf.sail.BigdataSail$BigdataSailConnection.flushStatementBuffers(BigdataSail.java:4183) at com.bigdata.rdf.sail.BigdataSail$BigdataSailConnection.commit2(BigdataSail.java:3925) at com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit2(BigdataSailRepositoryConnection.java:330) at com.bigdata.rdf.sparql.ast.eval.AST2BOpUpdate.convertCommit(AST2BOpUpdate.java:375) at com.bigdata.rdf.sparql.ast.eval.AST2BOpUpdate.convertUpdate(AST2BOpUpdate.java:321) at com.bigdata.rdf.sparql.ast.eval.ASTEvalHelper.executeUpdate(ASTEvalHelper.java:1072) ... 9 more

And here is the exception using HTTPRepository:
org.openrdf.repository.http.HTTPUpdateExecutionException: SPARQL-UPDATE: updateStr=prefix skos: <http://www.w3.org/2004/02/skos/core#> prefix skosxl: <http://www.w3.org/2008/05/skos-xl#> prefix dcterms: <http://purl.org/dc/terms/> prefix semwebo: <http://semweb.ch/onto/> DELETE { <http://semweb.ch/lodowl/0> skos:prefLabel ?x } WHERE {<http://semweb.ch/lodowl/0> skos:prefLabel ?x} java.util.concurrent.ExecutionException: java.util.concurrent.ExecutionException: org.openrdf.query.UpdateExecutionException: java.lang.NullPointerException at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at com.bigdata.rdf.sail.webapp.BigdataServlet.submitApiTask(BigdataServlet.java:281) at com.bigdata.rdf.sail.webapp.QueryServlet.doSparqlUpdate(QueryServlet.java:458) at com.bigdata.rdf.sail.webapp.QueryServlet.doPost(QueryServlet.java:239) at com.bigdata.rdf.sail.webapp.RESTServlet.doPost(RESTServlet.java:269) at com.bigdata.rdf.sail.webapp.MultiTenancyServlet.doPost(MultiTenancyServlet.java:193) at javax.servlet.http.HttpServlet.service(HttpServlet.java:648) at javax.servlet.http.HttpServlet.service(HttpServlet.java:729) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:230) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:108) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:522) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:1096) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:760) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1480) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745) Caused by: java.util.concurrent.ExecutionException: org.openrdf.query.UpdateExecutionException: java.lang.NullPointerException at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at com.bigdata.rdf.sail.webapp.QueryServlet$SparqlUpdateTask.call(QueryServlet.java:569) at com.bigdata.rdf.sail.webapp.QueryServlet$SparqlUpdateTask.call(QueryServlet.java:470) at com.bigdata.rdf.task.ApiTaskForIndexManager.call(ApiTaskForIndexManager.java:68) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ... 1 more Caused by: org.openrdf.query.UpdateExecutionException: java.lang.NullPointerException at com.bigdata.rdf.sparql.ast.eval.ASTEvalHelper.executeUpdate(ASTEvalHelper.java:1080) at com.bigdata.rdf.sail.BigdataSailUpdate.execute2(BigdataSailUpdate.java:152) at com.bigdata.rdf.sail.webapp.BigdataRDFContext$UpdateTask.doQuery(BigdataRDFContext.java:1968) at com.bigdata.rdf.sail.webapp.BigdataRDFContext$AbstractQueryTask.innerCall(BigdataRDFContext.java:1569) at com.bigdata.rdf.sail.webapp.BigdataRDFContext$AbstractQueryTask.call(BigdataRDFContext.java:1534) at com.bigdata.rdf.sail.webapp.BigdataRDFContext$AbstractQueryTask.call(BigdataRDFContext.java:747) ... 4 more Caused by: java.lang.NullPointerException at com.bigdata.rdf.inf.FullyBufferedJustificationIterator.<init>(FullyBufferedJustificationIterator.java:98) at com.bigdata.rdf.inf.Justification.isGrounded(Justification.java:809) at com.bigdata.rdf.inf.Justification.isGrounded(Justification.java:641) at com.bigdata.rdf.inf.TruthMaintenance.retractAll(TruthMaintenance.java:850) at com.bigdata.rdf.inf.TruthMaintenance.retractAll(TruthMaintenance.java:612) at com.bigdata.rdf.sail.BigdataSail$BigdataSailConnection.flushStatementBuffers(BigdataSail.java:4183) at com.bigdata.rdf.sail.BigdataSail$BigdataSailConnection.commit2(BigdataSail.java:3925) at com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit2(BigdataSailRepositoryConnection.java:330) at com.bigdata.rdf.sparql.ast.eval.AST2BOpUpdate.convertCommit(AST2BOpUpdate.java:375) at com.bigdata.rdf.sparql.ast.eval.AST2BOpUpdate.convertUpdate(AST2BOpUpdate.java:321) at com.bigdata.rdf.sparql.ast.eval.ASTEvalHelper.executeUpdate(ASTEvalHelper.java:1072) ... 9 more at org.openrdf.repository.http.HTTPUpdate.execute(HTTPUpdate.java:79) at com.semweb.sesame.SESAME_repository.update_tuples(SESAME_repository.java:471) at com.semweb.skosshuttle.engine.S2MEngine.create_newconceptscheme_in_remote_store(S2MEngine.java:5196) at com.semweb.skosshuttle.engine.S2MEngineActions.on_createconceptscheme(S2MEngineActions.java:2210) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at com.semweb.skosshuttle.engine.S2MEngineActions.doGet(S2MEngineActions.java:578) at javax.servlet.http.HttpServlet.service(HttpServlet.java:622) at javax.servlet.http.HttpServlet.service(HttpServlet.java:729) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:230) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:108) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:522) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:1096) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:760) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1480) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745) Caused by: org.openrdf.repository.RepositoryException: SPARQL-UPDATE: updateStr=prefix skos: <http://www.w3.org/2004/02/skos/core#> prefix skosxl: <http://www.w3.org/2008/05/skos-xl#> prefix dcterms: <http://purl.org/dc/terms/> prefix semwebo: <http://semweb.ch/onto/> DELETE { <http://semweb.ch/lodowl/0> skos:prefLabel ?x } WHERE {<http://semweb.ch/lodowl/0> skos:prefLabel ?x} java.util.concurrent.ExecutionException: java.util.concurrent.ExecutionException: org.openrdf.query.UpdateExecutionException: java.lang.NullPointerException at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at com.bigdata.rdf.sail.webapp.BigdataServlet.submitApiTask(BigdataServlet.java:281) at com.bigdata.rdf.sail.webapp.QueryServlet.doSparqlUpdate(QueryServlet.java:458) at com.bigdata.rdf.sail.webapp.QueryServlet.doPost(QueryServlet.java:239) at com.bigdata.rdf.sail.webapp.RESTServlet.doPost(RESTServlet.java:269) at com.bigdata.rdf.sail.webapp.MultiTenancyServlet.doPost(MultiTenancyServlet.java:193) at javax.servlet.http.HttpServlet.service(HttpServlet.java:648) at javax.servlet.http.HttpServlet.service(HttpServlet.java:729) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:230) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:108) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:522) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:1096) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:760) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1480) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745) Caused by: java.util.concurrent.ExecutionException: org.openrdf.query.UpdateExecutionException: java.lang.NullPointerException at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at com.bigdata.rdf.sail.webapp.QueryServlet$SparqlUpdateTask.call(QueryServlet.java:569) at com.bigdata.rdf.sail.webapp.QueryServlet$SparqlUpdateTask.call(QueryServlet.java:470) at com.bigdata.rdf.task.ApiTaskForIndexManager.call(ApiTaskForIndexManager.java:68) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ... 1 more Caused by: org.openrdf.query.UpdateExecutionException: java.lang.NullPointerException at com.bigdata.rdf.sparql.ast.eval.ASTEvalHelper.executeUpdate(ASTEvalHelper.java:1080) at com.bigdata.rdf.sail.BigdataSailUpdate.execute2(BigdataSailUpdate.java:152) at com.bigdata.rdf.sail.webapp.BigdataRDFContext$UpdateTask.doQuery(BigdataRDFContext.java:1968) at com.bigdata.rdf.sail.webapp.BigdataRDFContext$AbstractQueryTask.innerCall(BigdataRDFContext.java:1569) at com.bigdata.rdf.sail.webapp.BigdataRDFContext$AbstractQueryTask.call(BigdataRDFContext.java:1534) at com.bigdata.rdf.sail.webapp.BigdataRDFContext$AbstractQueryTask.call(BigdataRDFContext.java:747) ... 4 more Caused by: java.lang.NullPointerException at com.bigdata.rdf.inf.FullyBufferedJustificationIterator.<init>(FullyBufferedJustificationIterator.java:98) at com.bigdata.rdf.inf.Justification.isGrounded(Justification.java:809) at com.bigdata.rdf.inf.Justification.isGrounded(Justification.java:641) at com.bigdata.rdf.inf.TruthMaintenance.retractAll(TruthMaintenance.java:850) at com.bigdata.rdf.inf.TruthMaintenance.retractAll(TruthMaintenance.java:612) at com.bigdata.rdf.sail.BigdataSail$BigdataSailConnection.flushStatementBuffers(BigdataSail.java:4183) at com.bigdata.rdf.sail.BigdataSail$BigdataSailConnection.commit2(BigdataSail.java:3925) at com.bigdata.rdf.sail.BigdataSailRepositoryConnection.commit2(BigdataSailRepositoryConnection.java:330) at com.bigdata.rdf.sparql.ast.eval.AST2BOpUpdate.convertCommit(AST2BOpUpdate.java:375) at com.bigdata.rdf.sparql.ast.eval.AST2BOpUpdate.convertUpdate(AST2BOpUpdate.java:321) at com.bigdata.rdf.sparql.ast.eval.ASTEvalHelper.executeUpdate(ASTEvalHelper.java:1072) ... 9 more at org.openrdf.http.client.HTTPClient.handleHTTPError(HTTPClient.java:953) at org.openrdf.http.client.HTTPClient.sendUpdate(HTTPClient.java:390) at org.openrdf.repository.http.HTTPUpdate.execute(HTTPUpdate.java:57) ... 31 more

presigned s3 url of *.gz will not get load

For instance , consider this url
https://some-bucket.s3.amazonaws.com/triples-file.nt.gz?x-amz-security-token=XXX&AWSAccessKeyId=XXXX&Expires=1463756979&Signature=XXXX

The following sparql update load <https://some-bucket.s3.amazonaws.com/triples-file.nt.gz?x-amz-security-token=XXX&AWSAccessKeyId=XXXX&Expires=1463756979&Signature=XXXX> will load 0 triples .

I tracked this down to

. Note how the code is is use n.endsWith predicate to decide whether to use gzipInputStream . In the case of this url , n is triples-file.nt.gz?x-amz-security-token=XXX&AWSAccessKeyId=XXXX&Expires=1463756979&Signature=XXXX .

I think this is a bug

Java POST returns a 400

Dear Blazegraph community - (sorry for formatting, I cannot get it better...)

using bigdata
Tue Jan 24 18:03:47 CET 2017
Mac OS X/10.11.6 x86_64
Mac OS X 10.11.6 x86_64 #CPU=8
Oracle Corporation 1.8.0_25
freeMemory=137061912
buildVersion=2.1.4
gitCommit=738d05f08cffd319233a4bfbb0ec2a858e260f9c

I am having some issues transmitting a POST request with a SPARQL-UPDATE query inside using Java. Yes, I am using HttpURLConnection to post the query to Blazegraph.

Suppose http://13.0.2.130:8034/bigdata/namespace/demoEXTERNALULTIREP/sparql be a valid repo URL, sending even a simple query returns in the Java code an HTTP 400

String contenttype="application/x-www-form-urlencoded; charset=UTF-8";
String accept = "/";
String acceptlanguage = "en-US,de;q=0.7,en;q=0.3";
String acceptencoding = "gzip, deflate";
urlquery="http://13.0.2.130:8034/bigdata/namespace/demoEXTERNALULTIREP/sparql";

data = "query=INSERT {http://xx.ch/lod/base a "test"} WHERE {}";
URL myURL = new URL(urlquery);
String host = myURL.getHost()+":"+(myURL.getPort()!=80?myURL.getPort():"");
HttpURLConnection urlconn = (HttpURLConnection)myURL.openConnection();
urlconn.setDoInput(true);
urlconn.setDoOutput(true);
urlconn.setRequestMethod("POST");
if (!user.equals("") && !passwd.equals(""))
{
String userCredentials = user+":"+passwd;
String basicAuth = "Basic " + new String(S2MP.b46encode(userCredentials.getBytes()));
urlconn.setRequestProperty ("Authorization", basicAuth);
}
urlconn.setRequestProperty("Host", host);
urlconn.setRequestProperty("Accept", accept);
urlconn.setRequestProperty("Accept-Language", acceptlanguage);
urlconn.setRequestProperty("Accept-Encoding", acceptencoding);
urlconn.setRequestProperty("Content-Type", contenttype);
urlconn.setRequestProperty("Content-Length", String.valueOf(data.length()));

		urlconn.setUseCaches(false);
		urlconn.setConnectTimeout(10000);
		urlconn.setReadTimeout(10000);
		urlconn.connect();
		// myURLConnection.getOutputStream().write(bytes);
		OutputStreamWriter out = new OutputStreamWriter(urlconn.getOutputStream());
		out.write(data);
		out.close();
        int HttpResult = urlconn.getResponseCode();
        if (HttpResult == HttpURLConnection.HTTP_OK) {
			InputStream is = urlconn.getInputStream();
			BufferedReader reader = new BufferedReader( new InputStreamReader( is )  );
			reader.close();
        }

This code does not accomplish an UPDATE (no data is updated).
The same code but with "GET" returns an HTTP 200.

What should I do in order to be able to post this SPARQL-UPDATE query correctly ?

Thanks a lot in advance!

PS: To say the truth, this Java code returns a 400 even if I do not send any data.

PS2: Blazegraph threw the following exception (although the same query inside the workbench is adding one statement):

ERROR: BigdataRDFServlet.java:214: cause=java.util.concurrent.ExecutionException: org.openrdf.query.MalformedQueryException: Encountered " "insert" "INSERT "" at line 1, column 1.
Was expecting one of:
"base" ...
"prefix" ...
"select" ...
"construct" ...
"describe" ...
"ask" ...
, query=SPARQL-QUERY: queryStr=INSERT {http://xx.ch/lod/base a "test"} WHERE {}
java.util.concurrent.ExecutionException: org.openrdf.query.MalformedQueryException: Encountered " "insert" "INSERT "" at line 1, column 1.
Was expecting one of:
"base" ...
"prefix" ...
"select" ...
"construct" ...
"describe" ...
"ask" ...

at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at com.bigdata.rdf.sail.webapp.BigdataServlet.submitApiTask(BigdataServlet.java:281)
at com.bigdata.rdf.sail.webapp.QueryServlet.doSparqlQuery(QueryServlet.java:653)
at com.bigdata.rdf.sail.webapp.QueryServlet.doPost(QueryServlet.java:273)
at com.bigdata.rdf.sail.webapp.RESTServlet.doPost(RESTServlet.java:269)
at com.bigdata.rdf.sail.webapp.MultiTenancyServlet.doPost(MultiTenancyServlet.java:193)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:648)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:230)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:108)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:522)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:1096)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:760)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1480)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)

Caused by: org.openrdf.query.MalformedQueryException: Encountered " "insert" "INSERT "" at line 1, column 1.
Was expecting one of:
"base" ...
"prefix" ...
"select" ...
"construct" ...
"describe" ...
"ask" ...

at com.bigdata.rdf.sail.sparql.Bigdata2ASTSPARQLParser.parseQuery2(Bigdata2ASTSPARQLParser.java:400)
at com.bigdata.rdf.sail.webapp.QueryServlet$SparqlQueryTask.call(QueryServlet.java:716)
at com.bigdata.rdf.sail.webapp.QueryServlet$SparqlQueryTask.call(QueryServlet.java:670)
at com.bigdata.rdf.task.ApiTaskForIndexManager.call(ApiTaskForIndexManager.java:68)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
... 1 more

Caused by: com.bigdata.rdf.sail.sparql.ast.ParseException: Encountered " "insert" "INSERT "" at line 1, column 1.
Was expecting one of:
"base" ...
"prefix" ...
"select" ...
"construct" ...
"describe" ...
"ask" ...

at com.bigdata.rdf.sail.sparql.ast.SyntaxTreeBuilder.generateParseException(SyntaxTreeBuilder.java:9718)
at com.bigdata.rdf.sail.sparql.ast.SyntaxTreeBuilder.jj_consume_token(SyntaxTreeBuilder.java:9585)
at com.bigdata.rdf.sail.sparql.ast.SyntaxTreeBuilder.Query(SyntaxTreeBuilder.java:337)
at com.bigdata.rdf.sail.sparql.ast.SyntaxTreeBuilder.QueryContainer(SyntaxTreeBuilder.java:212)
at com.bigdata.rdf.sail.sparql.ast.SyntaxTreeBuilder.parseQuery(SyntaxTreeBuilder.java:32)
at com.bigdata.rdf.sail.sparql.Bigdata2ASTSPARQLParser.parseQuery2(Bigdata2ASTSPARQLParser.java:336)
... 7 more

location of query logs

Hello,
I've just installed the latest version of blazegraph on Windows. From the workbench, when I make a query, I can see some logs info such as the time, query, results, execution time. I was wondering where can I get the same info in the logs? I was looking my install repo but didn't find any logs. TIA

BulkLoader seems to ignore ".xml" extension files

I've deb-installed the 2.1 version of blazegraph and also downloaded the blazegraph.jar bundle file.

I'm trying to bulk load a directory containing a bunch of files with a ".xml" file extension with the following command line:

java -cp blazegraph.jar com.bigdata.rdf.store.DataLoader fastload.properties mydir

and nothing happens.

If I rename each ".xml" file into ".rdf", files get imported. Is it the expected behavior or did I miss something ?

Thanks for this great project!

Regards,
Adrien

war file: bigdata.jnl not found

Hi,

I downloaded the release war-file and tried to deploy it on tomcat.

However, the workbench does not load and in the tomcat logs I get
that file bigdata.jnl cannot be accessed. I searched for the file but it doesn't seem to exist on the server.
Do I have to also install the rpm/dep or anything else?

Is there a file where blazegraph writes own logs to?

Can someone point me to some useful documentation for running blazegraph with tomcat?

Thanks!
Martin

Here is the exception stack:
SEVERE: Exception sending context initialized event to listener instance of class com.bigdata.rdf.sail.webapp.BigdataRDFServletContextListener java.lang.RuntimeException: java.lang.RuntimeException: file=bigdata.jnl at com.bigdata.rdf.sail.webapp.BigdataRDFServletContextListener.openIndexManager(BigdataRDFServletContextListener.java:805) at com.bigdata.rdf.sail.webapp.BigdataRDFServletContextListener.contextInitialized(BigdataRDFServletContextListener.java:277) at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4992) at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5490) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150) at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:901) at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:877) at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:649) at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:1083) at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1880) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.RuntimeException: file=bigdata.jnl at com.bigdata.journal.FileMetadata.<init>(FileMetadata.java:851) at com.bigdata.journal.FileMetadata.createInstance(FileMetadata.java:1513) at com.bigdata.journal.AbstractJournal.<init>(AbstractJournal.java:1156) at com.bigdata.journal.Journal.<init>(Journal.java:276) at com.bigdata.journal.Journal.<init>(Journal.java:269) at com.bigdata.rdf.sail.webapp.BigdataRDFServletContextListener.openIndexManager(BigdataRDFServletContextListener.java:799) ... 14 more Caused by: java.io.FileNotFoundException: bigdata.jnl (Permission denied) at java.io.RandomAccessFile.open0(Native Method) at java.io.RandomAccessFile.open(RandomAccessFile.java:316) at java.io.RandomAccessFile.<init>(RandomAccessFile.java:243) at com.bigdata.journal.FileMetadata.reopenChannel(FileMetadata.java:1195) at com.bigdata.journal.FileMetadata.access$000(FileMetadata.java:58) at com.bigdata.journal.FileMetadata$1.reopenChannel(FileMetadata.java:1163) at com.bigdata.journal.FileMetadata$1.reopenChannel(FileMetadata.java:1153) at com.bigdata.journal.FileMetadata.<init>(FileMetadata.java:464) ... 19 more

CLEANLY ADD & DELETE RDF ONTOLOGIES ?

Dear blazegraph community / SYSTAP

Semweb realized several applications successfully using blazegraph. Thank you for being.
I am using the latest version of your blazegraph.jar (thank you).

What I am rather sceptical of is cleanly ADDING / DELETING RDF ontologies.
An ontology - like for instance http://spec.edmcouncil.org/fibo/FND/Parties/Roles/ contains structures which - once read in into the repository - ends up in blank nodes. It is important for me to add an ontology to a (now blazegraph) repository with truth maintenance, then on demand delete that ontology.

Sesame 2.8 (used and customized by SYSTAP for blazegraph) allows a file-input via connection.add() in a remoterepository but has no way to delete the same RDF file using the same connection (asymmetry of sesame 2.8); using connection.remove(statement) does not work with remoterepository (throws an exception). Blazegraph wants to remain relied on sesame 2.8 because it is used by most of its customers. It is not even known whether SYSTAP will ever produce a sesame 4 based blazegraph (up to now and up to "official" annoucements).

So I searched for another way to add/delete RDF Ontology material to/from a blazegraph journal: via SPARQLUPDATE. An ontology is firstly loaded into a sesame model (included in blazegraph sesame), then each (NT) statement is collected and written to a string, then an INSERT statement is built using those NT Triples and those triples are (successfully) inserted via SPARQLUPDATE. So far so good.

THE PROBLEM: The same added triples ARE NOT DELETABLE via the same SPARQLUPDATE (simply turning "DELETE" into "INSERT" in the same SPARQLUPDATE tab inside blazegraph workbench - same behavior as in my java code), because those triples contains blank nodes (says blazegraph in an exception).

So blazegraph seems to be not able to DELETE cleanly RDF material ... and this is besides scalability and greatest performance (...) stated by your selling department, a big problem.

How is this possible and is it so wanted/needed/planned?

However I hope I ran myself into some mistakes and hope someone can help solve this issue ...
Attached the statements which are INSERTABLE but not DELETABLE.

Thank you
Regards

Some attachment to help you finding a solution (please open a ticket in case you need it):
insertable_notdeletable_statements.txt
deletingexception_blazegraph.txt

Unable to install rpm package

Most of rpm-based (I have tested CentOS 6.9, RHEL 7.4 and Fedora 26) distros have "groupadd" command, instead of "addgroup", which used in preinstall script, so installation fails:

Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
/var/tmp/rpm-tmp.hE52e6: line 32: addgroup: command not found
error: %pre(blazegraph-rpm-2.1.1-SNAPSHOT.noarch) scriptlet failed, exit status 127
Error in PREIN scriptlet in rpm package blazegraph-rpm-2.1.1-SNAPSHOT.noarch
  Verifying  : blazegraph-rpm-2.1.1-SNAPSHOT.noarch                                                                                                                                                                                  1/1 

Failed:
  blazegraph-rpm.noarch 0:2.1.1-SNAPSHOT

IllegalStateException: Form too large when adding ontology

Hello Blazegraph Team,

I want to add an ontology file of about 125MB into Blazegraph Workbench (Version 2.1.5_RC).
When I do so, the workbench tells me ERROR: Could not contact server and I see two warnings on the console.

It seems similar to #8 but I'm using just the JAR from the console.

As long as the file is <100MB it works but when its larger there is a message that I have to enter the path manually and it won't work.

Blazegraph Status Build Version=2.1.5-SNAPSHOT Build Git Commit=1dad2d4cf1261a271d3766947be8916a7d7df761 Build Git Branch=refs/heads/BLAZEGRAPH_RELEASE_2_1_5_RC Accepted query count=0 Running query count=0 Show queries, query details.

/GeoSpatial/bigMinCalculationTimeMS=0
/GeoSpatial/filterCalculationTimeMS=0
/GeoSpatial/geoSpatialSearchRequests=0
/GeoSpatial/geoSpatialServiceCallSubRangeTasks=0
/GeoSpatial/geoSpatialServiceCallTasks=0
/GeoSpatial/rangeCheckCalculationTimeMS=0
/GeoSpatial/zOrderIndexHitRatio=null
/GeoSpatial/zOrderIndexHits=0
/GeoSpatial/zOrderIndexMisses=0
/GeoSpatial/zOrderIndexScannedValues=0
/blockedWorkQueueCount=0
/blockedWorkQueueRunningTotal=0
/bufferedChunkMessageBytesOnNativeHeap=0
/bufferedChunkMessageCount=0
/deadlineQueueSize=0
/operatorActiveCount=0
/operatorHaltCount=0
/operatorStartCount=0
/operatorTasksPerQuery=0.0
/queriesPerSecond=0.0
/queryDoneCount=0
/queryErrorCount=0
/queryStartCount=0

Although this seems like a Jetty problem (the stack trace is below), maybe you can send several requests so that this problem doesn't occur.

Stack Trace WARN : ServletHandler.java:626: /blazegraph/namespace/test/sparql java.lang.IllegalStateException: Form too large: 35166271 > 10485760 at org.eclipse.jetty.server.Request.extractFormParameters(Request.java:3 64) at org.eclipse.jetty.server.Request.extractContentParameters(Request.jav a:302) at org.eclipse.jetty.server.Request.extractParameters(Request.java:256) at org.eclipse.jetty.server.Request.getParameter(Request.java:827) at com.bigdata.rdf.sail.webapp.RESTServlet.doPost(RESTServlet.java:256) at com.bigdata.rdf.sail.webapp.MultiTenancyServlet.doPost(MultiTenancySe rvlet.java:193) at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:769 ) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java :585) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.j ava:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.jav a:577) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandl er.java:223) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandl er.java:1125) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java: 515) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandle r.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandle r.java:1059) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.j ava:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(Cont extHandlerCollection.java:215) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerColl ection.java:110) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper .java:97) at org.eclipse.jetty.server.Server.handle(Server.java:497) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.jav a:248) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java :540) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPoo l.java:610) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool .java:539) at java.lang.Thread.run(Unknown Source) WARN : HttpParser.java:1358: badMessage: java.lang.IllegalStateException: too mu ch data after closed for HttpChannelOverHttp@24cb80e8{r=1,c=false,a=IDLE,uri=-} WARN : ServletHandler.java:626: /blazegraph/namespace/test/sparql java.lang.IllegalStateException: Form too large: 35166271 > 10485760 at org.eclipse.jetty.server.Request.extractFormParameters(Request.java:3 64) at org.eclipse.jetty.server.Request.extractContentParameters(Request.jav a:302) at org.eclipse.jetty.server.Request.extractParameters(Request.java:256) at org.eclipse.jetty.server.Request.getParameter(Request.java:827) at com.bigdata.rdf.sail.webapp.RESTServlet.doPost(RESTServlet.java:256) at com.bigdata.rdf.sail.webapp.MultiTenancyServlet.doPost(MultiTenancySe rvlet.java:193) at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:769 ) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java :585) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.j ava:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.jav a:577) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandl er.java:223) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandl er.java:1125) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java: 515) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandle r.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandle r.java:1059) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.j ava:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(Cont extHandlerCollection.java:215) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerColl ection.java:110) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper .java:97) at org.eclipse.jetty.server.Server.handle(Server.java:497) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.jav a:248) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java :540) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPoo l.java:610) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool .java:539) at java.lang.Thread.run(Unknown Source) WARN : HttpParser.java:1358: badMessage: java.lang.IllegalStateException: too mu ch data after closed for HttpChannelOverHttp@5f454652{r=1,c=false,a=IDLE,uri=-} WARN : ServletHandler.java:626: /blazegraph/namespace/test/sparql java.lang.IllegalStateException: Form too large: 35166261 > 10485760 at org.eclipse.jetty.server.Request.extractFormParameters(Request.java:3 64) at org.eclipse.jetty.server.Request.extractContentParameters(Request.jav a:302) at org.eclipse.jetty.server.Request.extractParameters(Request.java:256) at org.eclipse.jetty.server.Request.getParameter(Request.java:827) at com.bigdata.rdf.sail.webapp.RESTServlet.doPost(RESTServlet.java:256) at com.bigdata.rdf.sail.webapp.MultiTenancyServlet.doPost(MultiTenancySe rvlet.java:193) at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:769 ) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java :585) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.j ava:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.jav a:577) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandl er.java:223) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandl er.java:1125) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java: 515) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandle r.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandle r.java:1059) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.j ava:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(Cont extHandlerCollection.java:215) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerColl ection.java:110) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper .java:97) at org.eclipse.jetty.server.Server.handle(Server.java:497) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.jav a:248) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java :540) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPoo l.java:610) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool .java:539) at java.lang.Thread.run(Unknown Source) WARN : HttpParser.java:1358: badMessage: java.lang.IllegalStateException: too much data after closed for HttpChannelOverHttp@1e784c69{r=1,c=false,a=IDLE,uri=-}

HTTPRepository strikes on "CONSTRUCT"

Hi

I am using HTTPRepository (since one year now) to pass query to remote Blazegraph repositories and get results in java using the SAIL interface in sesame 2.8 (blazegraph jar). All running very well except for the case that the query contains a "CONSTRUCT" statement. In that case, instead of returning results (there are results), the methods

TupleQuery tupleQuery = con.prepareTupleQuery(QueryLanguage.SPARQL, query); result = tupleQuery.evaluate();

throws an exception with message:

Server responded with an unsupported file format: application/rdf+xml org.openrdf.repository.http.HTTPQueryEvaluationException: Server responded with an unsupported file format: application/rdf+xml at org.openrdf.repository.http.HTTPTupleQuery.evaluate(HTTPTupleQuery.java:59)

The operations SELECT, DELETE, INSERT work as aspected. Just CONSTRUCT does not want.

What should I do in order to let this run ? Is it a bug ? Can I solve this (how) ?

Thanks

No obvious start-point for compilation and installation

Unlike just about every other non-trivial project I've seen, there are no indications of instructions on how to start compiling/building from the given source code. The instructions should be in the code base and not just on a wiki. Please at least provide a file in the top-level directory. (There appears to be something in older release data, so maybe the current release file should have this info).

Slow DataLoader Performance

I've bee trying various ways to load in a ~677 million triple .nt file with no luck. My last attempt was to use linux split to parse the files in 90k triples per file and run the DataLoader from command line. Everything seems fine at the start ~23k triples per sec loading. Then several hours later there is only a trickle of triples going through. I saw the same performance problems without splitting the file.

I'm using the latest blazegraph.jar file from sourceforge. Here is the command-line, properties file and partial terminal output.

Any help/suggestions would be appreciated. Thanks!

***************** command line *************************
java -cp *:*.jar com.bigdata.rdf.store.DataLoader -verbose -durableQueues /home/stabiledev/viaf/fastload.properties /home/stabiledev/viaf/batch &

****************** properties file **************************

# This configuration turns off incremental inference for load and retract, so
# you must explicitly force these operations if you want to compute the closure
# of the knowledge base.  Forcing the closure requires punching through the SAIL
# layer.  Of course, if you are not using inference then this configuration is
# just the ticket and is quite fast.

# set the journal file
com.bigdata.journal.AbstractJournal.file=/home/stabiledev/viaf/try2/viaf_data.jnl

# set the initial and maximum extent of the journal
com.bigdata.journal.AbstractJournal.initialExtent=209715200
com.bigdata.journal.AbstractJournal.maximumExtent=209715200

# turn off automatic inference in the SAIL
com.bigdata.rdf.sail.truthMaintenance=false

# don't store justification chains, meaning retraction requires full manual 
# re-closure of the database
com.bigdata.rdf.store.AbstractTripleStore.justify=false

# turn off the statement identifiers feature for provenance
com.bigdata.rdf.store.AbstractTripleStore.statementIdentifiers=false

# turn off the free text index
com.bigdata.rdf.store.AbstractTripleStore.textIndex=false

# RWStore (scalable single machine backend)
com.bigdata.journal.AbstractJournal.bufferMode=DiskRW

# Turn closure off
com.bigdata.rdf.store.DataLoader.closure=None



******************* teminal output ************************************


Reading properties: /home/stabiledev/viaf/fastload.properties
Will load from: /home/stabiledev/viaf/batch
Journal file: /home/stabiledev/viaf/try2/viaf_data.jnl
WARN : AbstractBTree.java:2167: Bloom filter disabled - maximum error rate would be exceeded: entryCount=1883228, factory=BloomFilterFactory{ n=1000000, p=0.02, maxP=0.15, maxN=1883227}
loading: 14130000 stmts added in 610.778 secs, rate= 23134, commitLatency=0ms, {failSet=0,goodSet=156}
loading: 22410000 stmts added in 1217.57 secs, rate= 18405, commitLatency=0ms, {failSet=0,goodSet=248}
loading: 28170000 stmts added in 1819.2 secs, rate= 15484, commitLatency=0ms, {failSet=0,goodSet=312}
loading: 34380000 stmts added in 2420.144 secs, rate= 14205, commitLatency=0ms, {failSet=0,goodSet=381}
loading: 38340000 stmts added in 3032.121 secs, rate= 12644, commitLatency=0ms, {failSet=0,goodSet=425}
WARN : AbstractBTree.java:3758: wrote: name=kb.spo.POS, 1 records (#nodes=1, #leaves=0) in 5211ms : addrRoot=-145370130275106556
loading: 43380000 stmts added in 3642.039 secs, rate= 11910, commitLatency=0ms, {failSet=0,goodSet=481}
WARN : AbstractBTree.java:3758: wrote: name=kb.spo.POS, 4 records (#nodes=1, #leaves=3) in 5247ms : addrRoot=-74144699035680380
WARN : AbstractBTree.java:3758: wrote: name=kb.spo.OSP, 1 records (#nodes=1, #leaves=0) in 5248ms : addrRoot=-74144703330647628
loading: 47070000 stmts added in 4261.653 secs, rate= 11045, commitLatency=0ms, {failSet=0,goodSet=522}
loading: 50670000 stmts added in 4866.663 secs, rate= 10411, commitLatency=0ms, {failSet=0,goodSet=562}

...

loading: 96570000 stmts added in 12807.511 secs, rate= 7540, commitLatency=0ms, {failSet=0,goodSet=1072}
loading: 100620000 stmts added in 13421.764 secs, rate= 7496, commitLatency=0ms, {failSet=0,goodSet=1117}
WARN : AbstractBTree.java:3758: wrote: name=kb.spo.POS, 7 records (#nodes=3, #leaves=4) in 5026ms : addrRoot=-620060906650337000
WARN : AbstractBTree.java:3758: wrote: name=kb.spo.OSP, 3 records (#nodes=1, #leaves=2) in 6001ms : addrRoot=-701707543457562350
WARN : AbstractBTree.java:3758: wrote: name=kb.spo.POS, 9 records (#nodes=5, #leaves=4) in 6001ms : addrRoot=-127697761486241280
loading: 105750000 stmts added in 14024.126 secs, rate= 7540, commitLatency=0ms, {failSet=0,goodSet=1174}

...

loading: 144720000 stmts added in 28388.47 secs, rate= 5097, commitLatency=0ms, {failSet=0,goodSet=1607}
loading: 145530000 stmts added in 29031.825 secs, rate= 5012, commitLatency=0ms, {failSet=0,goodSet=1616}

...

loading: 171360000 stmts added in 57245.59 secs, rate= 2993, commitLatency=0ms, {failSet=0,goodSet=1903}
loading: 171810000 stmts added in 57962.026 secs, rate= 2964, commitLatency=0ms, {failSet=0,goodSet=1908}

Ordered results in solution set - inconsistent results

Hi!
On our Blazegraph instances (version 2.1.2) we investigated usage of features regarding
ordered values in solution sets. It is use case described within,
https://wiki.blazegraph.com/wiki/index.php/SPARQL_Update#Named_Solution_Sets
which is used for paging through data set.
We ran into problems with this. For sample data in attachment we made solution set with:

prefix ex: <https://example.org/>
prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#>
prefix xml: <http://www.w3.org/XML/1998/namespace>
prefix xsd: <http://www.w3.org/2001/XMLSchema#>

INSERT INTO %entity_label
SELECT ?e ?e_l
WHERE {
    SELECT ?e ?e_l
    WHERE {
       ?e a ex:c .
       ?e rdfs:label ?e_l .
    }
ORDER BY ?e
}

and when queried with :

prefix ex: <https://example.org/>
SELECT ?e ?e_l
WHERE{
   INCLUDE %entity_label
 }
 LIMIT 3
 OFFSET 97

we get proper results which are:

<https://example.org/c/e/3eyl1r2> label 3eyl1r2
<https://example.org/c/e/3f06245> label 3f06245
<https://example.org/c/e/3funi1b> label 3funi1b

but not always. We also get wrong data like

<https://example.org/c/e/73fequm> label 73fequm
<https://example.org/c/e/73wns10> label 73wns10
<https://example.org/c/e/740peif> label 740peif

or else, from time to time.

Please, can you look at this problem? It breaks regular use case for above mentioned
feature described in link.

Best regards,
Adam
data.zip

JSON-LD Support

Is is possible to export a CONSTRUCT query from the Blazegraph NanoSparqlServer GUI or API (e.g., using an HTTP Accept Header) as JSON-LD? If not, is JSON-LD support on the roadmap?

GitHub Issues vs JIRA

The README.md reads:

"Find an issue? Need help? See JIRA or purchase Support."

Jira is available at https://jira.blazegraph.com/secure/Dashboard.jspa and the GitHub blazegraph/database repository has a Issues tab. Perhaps one of these two tools could be closed to new issues so that everything is consolidated, or guidance could be added to README.md so that it is clear which should be used in a given case.

BigdataGraph#addEdge with same SOP but different predicate creates LinkedList of labels, causing ClassCastException on BigdataEdge#getLabel

String graphUrl = "http://localhost:9200/bigdata/sparql";
BigdataGraph graph = new BigdataGraph(graphUrl);
Vertex subject = graph.addVertex("subject");
Vertex object = graph.addVertex("object");
String edgeId = UUID.randomUUID().toString();

String firstLabel = "label1";
Edge firstEdge = graph.addEdge(edgeId, subject, object, firstLabel)

String secondLabel = "label2";
Edge secondEdge = graph.addEdge(edgeId, subject, object, secondLabel);

// throws java.lang.ClassCastException: java.util.LinkedList cannot be cast to java.lang.String
secondEdge.getLabel();

2.1.4 release on Debian package installation - /usr/bin/loadRestAPI.sh seems to do nothing

The script (with a patch to fix the RWStore.properties for the 2.1.4 release) seems to run but does nothing, e.g.

$ /usr/bin/loadRestAPI.sh /data/src/dlss/ld4l/ld4p_root/rdf/MarcRDF
Loading with properties...
quiet=false
verbose=0
closure=false
durableQueues=true
#Needed for quads
#defaultGraph=
com.bigdata.rdf.store.DataLoader.flush=false
com.bigdata.rdf.store.DataLoader.bufferCapacity=100000
com.bigdata.rdf.store.DataLoader.queueCapacity=10
#Namespace to load
namespace=kb
#Files to load
fileOrDirs=/data/src/dlss/ld4l/ld4p_root/rdf/MarcRDF
#Property file (if creating a new namespace)
propertyFile=/etc/blazegraph/RWStore.properties
<?xml version="1.0"?><data modified="0" milliseconds="20"/>DATALOADER-SERVLET: Loaded kb with properties: /etc/blazegraph/RWStore.properties

After waiting a while, the WebUI Query results show nothing and also the files are not modified:

$ find /data/src/dlss/ld4l/ld4p_root/rdf/MarcRDF/ -type f -name '*.rdf' | wc -l
36074
$ find /data/src/dlss/ld4l/ld4p_root/rdf/MarcRDF/ -type f -name '*.rdf.good' | wc -l
0

The service seems to be running, based on the WebUI responses at localhost:9999 and the service blazegraph status seems OK. The logs in /var/log/blazegraph seem to be void of any useful information about the dataloader activity. The last entry is just

  • Reading properties: /etc/blazegraph/RWStore.properties

Full text reindexing doesn't work

Hi everybody
I was experimenting with the full text index of blazegraph and I've determined that the full text index doesn't work anymore after a reindex.
Steps to reproduce:

  • Start the blazegrapgh v2.1 server by java -server -Xmx4g -jar blazegraph.jar
  • Create a new namespace test and activate full text search. My config looks like this:
com.bigdata.rdf.store.AbstractTripleStore.axiomsClass=com.bigdata.rdf.axioms.NoAxioms
com.bigdata.rdf.sail.isolatableIndices=false
com.bigdata.rdf.store.AbstractTripleStore.justify=false
com.bigdata.rdf.sail.truthMaintenance=false
com.bigdata.rdf.sail.namespace=test
com.bigdata.rdf.store.AbstractTripleStore.quads=false
com.bigdata.namespace.test.spo.com.bigdata.btree.BTree.branchingFactor=1024
com.bigdata.namespace.test.lex.com.bigdata.btree.BTree.branchingFactor=400
com.bigdata.journal.Journal.groupCommit=false
com.bigdata.rdf.store.AbstractTripleStore.geoSpatial=false
com.bigdata.rdf.store.AbstractTripleStore.statementIdentifiers=false
  • Go to the update tab and insert some dummy data, e.g.:
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
INSERT DATA {
    <http://example.org/test> rdfs:label "Test property" .
}
  • When you query now the data with full text search, you will find a result:
PREFIX bds: <http://www.bigdata.com/rdf/search#>

SELECT * {
  ?o bds:search "test*" .
}
  • If you rebuild the full text index with the button on the namespaces tab (or recreate it in the console), the full text search doesn't work anymore, e.g. the same query as in the point above doesn't return any values.

Best idlefella

Blazegraph 2.1.4 debian package installed on Ubuntu 16.04 - fails to restart

Followed the instructions in the blazegraph-deb section to build and install 2.1.4 and it worked at first. But after trying to load a lot of data the system froze. On restart, blazegraph will not restart using sudo service blazegraph restart (although a following status indicates it is running OK). The log shows the failure to start is a com.bigdata.util.ChecksumError -- is this a bug or a feature?

INFO: com.bigdata.util.config.LogUtil: Configure and watch: /etc/blazegraph/log4j.properties

BlazeGraph(TM) Graph Engine

                   Flexible
                   Reliable
                  Affordable
      Web-Scale Computing for the Enterprise

Copyright SYSTAP, LLC DBA Blazegraph 2006-2016.  All rights reserved.

sul-dlweber-ubuntu
Fri Feb 17 20:59:16 PST 2017
Linux/4.4.0-62-generic amd64
Intel(R) Core(TM) i7-3720QM CPU @ 2.60GHz Family 6 Model 58 Stepping 9, GenuineIntel #CPU=2
Oracle Corporation 1.7.0_80
freeMemory=123735856
buildVersion=2.1.4
gitCommit=738d05f08cffd319233a4bfbb0ec2a858e260f9c

Dependency         License                                                                 
ICU                http://source.icu-project.org/repos/icu/icu/trunk/license.html          
bigdata-ganglia    http://www.apache.org/licenses/LICENSE-2.0.html                         
blueprints-core    https://github.com/tinkerpop/blueprints/blob/master/LICENSE.txt         
colt               http://acs.lbl.gov/software/colt/license.html                           
commons-codec      http://www.apache.org/licenses/LICENSE-2.0.html                         
commons-fileupload http://www.apache.org/licenses/LICENSE-2.0.html                         
commons-io         http://www.apache.org/licenses/LICENSE-2.0.html                         
commons-logging    http://www.apache.org/licenses/LICENSE-2.0.html                         
dsiutils           http://www.gnu.org/licenses/lgpl-2.1.html                               
fastutil           http://www.apache.org/licenses/LICENSE-2.0.html                         
flot               http://www.opensource.org/licenses/mit-license.php                      
high-scale-lib     http://creativecommons.org/licenses/publicdomain                        
httpclient         http://www.apache.org/licenses/LICENSE-2.0.html                         
httpclient-cache   http://www.apache.org/licenses/LICENSE-2.0.html                         
httpcore           http://www.apache.org/licenses/LICENSE-2.0.html                         
httpmime           http://www.apache.org/licenses/LICENSE-2.0.html                         
jackson-core       http://www.apache.org/licenses/LICENSE-2.0.html                         
jetty              http://www.apache.org/licenses/LICENSE-2.0.html                         
jquery             https://github.com/jquery/jquery/blob/master/MIT-LICENSE.txt            
jsonld             https://raw.githubusercontent.com/jsonld-java/jsonld-java/master/LICENCE
log4j              http://www.apache.org/licenses/LICENSE-2.0.html                         
lucene             http://www.apache.org/licenses/LICENSE-2.0.html                         
nanohttp           http://elonen.iki.fi/code/nanohttpd/#license                            
rexster-core       https://github.com/tinkerpop/rexster/blob/master/LICENSE.txt            
river              http://www.apache.org/licenses/LICENSE-2.0.html                         
semargl            https://github.com/levkhomich/semargl/blob/master/LICENSE               
servlet-api        http://www.apache.org/licenses/LICENSE-2.0.html                         
sesame             http://www.openrdf.org/download.jsp                                     
slf4j              http://www.slf4j.org/license.html                                       
zookeeper          http://www.apache.org/licenses/LICENSE-2.0.html                         

WARN : NanoSparqlServer.java:517: Starting NSS
WARN : WebAppContext.java:506: Failed startup of context o.e.j.w.WebAppContext@2db238ce{/blazegraph,file:/usr/share/blazegraph-2.1.4/war/,STARTING}{/usr/share/blazegraph/war/}
java.lang.RuntimeException: java.lang.RuntimeException: addr=-374049 : cause=com.bigdata.util.ChecksumError: offset=225590272,nbytes=426,expected=0,actual=27656193
	at com.bigdata.rdf.sail.webapp.BigdataRDFServletContextListener.openIndexManager(BigdataRDFServletContextListener.java:805)
	at com.bigdata.rdf.sail.webapp.BigdataRDFServletContextListener.contextInitialized(BigdataRDFServletContextListener.java:277)
	at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:798)
	at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444)
	at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:789)
	at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294)
	at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1341)
	at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1334)
	at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)
	at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:497)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
	at org.eclipse.jetty.server.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:163)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
	at org.eclipse.jetty.server.Server.start(Server.java:387)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
	at org.eclipse.jetty.server.Server.doStart(Server.java:354)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at com.bigdata.rdf.sail.webapp.NanoSparqlServer.awaitServerStart(NanoSparqlServer.java:518)
	at com.bigdata.rdf.sail.webapp.NanoSparqlServer.main(NanoSparqlServer.java:482)
Caused by: java.lang.RuntimeException: addr=-374049 : cause=com.bigdata.util.ChecksumError: offset=225590272,nbytes=426,expected=0,actual=27656193
	at com.bigdata.rwstore.RWStore.getData(RWStore.java:2097)
	at com.bigdata.journal.RWStrategy.readFromLocalStore(RWStrategy.java:732)
	at com.bigdata.journal.RWStrategy.read(RWStrategy.java:155)
	at com.bigdata.journal.AbstractJournal._getCommitRecord(AbstractJournal.java:4601)
	at com.bigdata.journal.AbstractJournal.<init>(AbstractJournal.java:1328)
	at com.bigdata.journal.Journal.<init>(Journal.java:276)
	at com.bigdata.journal.Journal.<init>(Journal.java:269)
	at com.bigdata.rdf.sail.webapp.BigdataRDFServletContextListener.openIndexManager(BigdataRDFServletContextListener.java:799)
	... 27 more
Caused by: com.bigdata.util.ChecksumError: offset=225590272,nbytes=426,expected=0,actual=27656193
	at com.bigdata.io.writecache.WriteCacheService._readFromLocalDiskIntoNewHeapByteBuffer(WriteCacheService.java:3783)
	at com.bigdata.io.writecache.WriteCacheService._getRecord(WriteCacheService.java:3598)
	at com.bigdata.io.writecache.WriteCacheService.access$2500(WriteCacheService.java:200)
	at com.bigdata.io.writecache.WriteCacheService$1.compute(WriteCacheService.java:3435)
	at com.bigdata.io.writecache.WriteCacheService$1.compute(WriteCacheService.java:3419)
	at com.bigdata.util.concurrent.Memoizer$1.call(Memoizer.java:77)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at com.bigdata.util.concurrent.Memoizer.compute(Memoizer.java:92)
	at com.bigdata.io.writecache.WriteCacheService.loadRecord(WriteCacheService.java:3540)
	at com.bigdata.io.writecache.WriteCacheService.read(WriteCacheService.java:3259)
	at com.bigdata.rwstore.RWStore.getData(RWStore.java:2088)
	... 34 more
serviceURL: http://172.17.0.1:9999

Silently ignores errors in prefix file

In PrefixDeclProcessor, the loading silently stops if it sees an empty line, even if there are more prefixes defined below. For this file, only the first prefix will be loaded:

PREFIX osmm: <https://www.openstreetmap.org/meta/>

PREFIX osmt: <https://wiki.openstreetmap.org/wiki/Key:>

To be consistent with other prop files, it should skip the blank lines and lines that begin with the "#" after trimming whitespace.

Also, possibly a separate issue, Blazegraph mishandles if the same URL is declared as two different prefixes. It shows that both are loaded, but then query engine fails to recognize it.

Switch from comparing qualified class to instanceof in BigdataElement#equals to allow subclassing.

I made a class to cache the edge label, subclassing from BigdataEdge (code below). I did this because I was getting a ClassCastException when I called BigdataEdge#getLabel when instantiating the BigdataEdge myself. However, BigdataEdge#equals always returns false when the other element is an instance of my subclass because it is comparing the results of getClass between the objects. It would be nice if the BigdataElement#equals method did an instanceof check instead; then equals would work for subclasses.

public class SimpleEdge extends BigdataEdge {

    private String label;

    public SimpleEdge(Statement stmt, BigdataGraph graph) {
        super(stmt, graph);
    }

    public SimpleEdge(Statement stmt, BigdataGraph graph, String label) {
        super(stmt, graph);
        this.label = label;
    }

    @Override
    public String getLabel() {
        return label;
    }
}

502 Proxy Error, when loading 2.6Gb file via the web interface

Hello,

I'm trying to load a 2.6Gb turtle file via the update panel of the web interface with the following command:

load <file:///mnt/data/a0001.ttl>

It gives me:

ERROR:

502 Proxy Error

Proxy Error
The proxy server received an invalid
response from an upstream server.
The proxy server could not handle the request POST /blazegraph/namespace/test/sparql.
Reason: Error reading from remote server

I'm not sure if it's a bug or if it's a problem related to my configuration.

After this error all triples seem to be correctly loaded.

use sparql will show:Another process that has locked a file part of the process can not be accessed

The code is below:
public static void main(String[] args) throws Exception {
final BigdataGraph graph = new BigdataGraphClient("http://localhost:9999/blazegraph/sparql");
for(int i = 0; i< 50; i++) {
Vertex v = graph.addVertex(null);
v.setProperty("name", "vertex" + i);
}
ICloseableIterator a = graph.select("select (COUNT(?s) AS ?count) where {?s ?p ?o}");
a.close();
graph.shutdown();
}
this code The first few times run is good,then will throw:Another process that has locked a file part of the process can not be accessed。

N-Triples files aren't read if blank node label starts withnumber

_:1 <http://example.org/p> "o".

the blank node identifier seems legal as https://www.w3.org/TR/n-triples/ says: "The characters _ and [0-9] may appear anywhere in a blank node label."

But inserting that in blazegrah results in:

ERROR: INSERT-WITH-BODY: baseURI=http://lindas.zazuko.com:9999/blazegraph/namespace/swisstopo/sparql, Content-Type=text/plain, context-uri=[] java.util.concurrent.ExecutionException: org.openrdf.rio.RDFParseException: Expected a letter, found: 1 [line 1] at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at com.bigdata.rdf.sail.webapp.BigdataServlet.submitApiTask(BigdataServlet.java:281) at com.bigdata.rdf.sail.webapp.InsertServlet.doPostWithBody(InsertServlet.java:203) at com.bigdata.rdf.sail.webapp.InsertServlet.doPost(InsertServlet.java:119) at com.bigdata.rdf.sail.webapp.RESTServlet.doPost(RESTServlet.java:308) at com.bigdata.rdf.sail.webapp.MultiTenancyServlet.doPost(MultiTenancyServlet.java:192) at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:769) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1125) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1059) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:497) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:248) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:610) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:539) at java.lang.Thread.run(Thread.java:745) Caused by: org.openrdf.rio.RDFParseException: Expected a letter, found: 1 [line 1] at org.openrdf.rio.helpers.RDFParserHelper.reportError(RDFParserHelper.java:348) at org.openrdf.rio.helpers.RDFParserBase.reportError(RDFParserBase.java:608) at org.openrdf.rio.ntriples.NTriplesParser.reportError(NTriplesParser.java:596) at org.openrdf.rio.ntriples.NTriplesParser.parseNodeID(NTriplesParser.java:455) at org.openrdf.rio.ntriples.NTriplesParser.parseSubject(NTriplesParser.java:343) at org.openrdf.rio.ntriples.NTriplesParser.parseTriple(NTriplesParser.java:282) at org.openrdf.rio.ntriples.NTriplesParser.parse(NTriplesParser.java:193) at org.openrdf.rio.ntriples.NTriplesParser.parse(NTriplesParser.java:132) at com.bigdata.rdf.sail.webapp.InsertServlet$InsertWithBodyTask.call(InsertServlet.java:305) at com.bigdata.rdf.sail.webapp.InsertServlet$InsertWithBodyTask.call(InsertServlet.java:226) at com.bigdata.rdf.task.ApiTaskForIndexManager.call(ApiTaskForIndexManager.java:68) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ... 1 more

move to rdf4j

Sesame is now renamed to rdf4j, it makes sense to gradually move to it

Blazegraph INFERENCE

Hello again dear community

Thank you so far for helping/answering my questions.
I am experiencing a more or less unexpected behaviour concerning inference while inserting "similar" (but not same) RDF Entities via SPARQL-UPDATE after having loaded an OWL ontology on Blazegraph 1.2.4 - the current one (thank you!).

The namespace options:

demo_TESTINFERENCE2

com.bigdata.rdf.store.AbstractTripleStore.justifytrue
com.bigdata.rdf.sail.namespacedemo_TESTINFERENCE2
com.bigdata.relation.classcom.bigdata.rdf.store.LocalTripleStore
com.bigdata.rdf.sail.truthMaintenancetrue
com.bigdata.btree.writeRetentionQueue.capacity16000
com.bigdata.rdf.store.AbstractTripleStore.textIndextrue
com.bigdata.search.FullTextIndex.fieldsEnabledfalse
com.bigdata.relation.namespacedemo_TESTINFERENCE2
com.bigdata.rdf.store.AbstractTripleStore.vocabularyClasscom.bigdata.rdf.vocab.core.BigdataCoreVocabulary_v20160317
com.bigdata.rdf.store.AbstractTripleStore.quadsfalse
com.bigdata.relation.containerdemo_TESTINFERENCE2
com.bigdata.journal.AbstractJournal.bufferModeDiskRW
com.bigdata.rdf.sail.bufferCapacity5000000

The behaviour is observed while updating (SPARQL-UPDATE) the namespace using twice the same query below with two different subjects (once with subject http://semweb.ch/base/9 and once with http://semweb.ch/base/19, also changing the prefLabel from "CCC9" to resp. "CCC19" to distinguish the entities also by a prefLabel (see below). This query just creates one concept scheme adding some attributes to it.

prefix skos: http://www.w3.org/2004/02/skos/core#
prefix skosxl: http://www.w3.org/2008/05/skos-xl#
prefix dcterms: http://purl.org/dc/terms/
prefix semwebo: http://semweb.ch/onto/
prefix rdf: http://www.w3.org/1999/02/22-rdf-syntax-ns#
prefix rdfs: http://www.w3.org/2000/01/rdf-schema#
prefix dc: http://purl.org/dc/elements/1.1/
prefix cco: http://creativecommons.org/ns#
prefix foaf: http://xmlns.com/foaf/0.1/
prefix void: http://rdfs.org/ns/void#
prefix nkos: http://purl.org/nkos/
INSERT {
http://semweb.ch/base/9 rdf:type skos:ConceptScheme.
http://semweb.ch/base/9 skos:prefLabel "CCC9"@en.
http://semweb.ch/base/9 dcterms:created "2016-11-27T13:09:14.332+01:00"^^http://www.w3.org/2001/XMLSchema#dateTime.
http://semweb.ch/base/9 dcterms:creator "Please specify".
http://semweb.ch/base/9 dcterms:subject "Please specify"@en.
http://semweb.ch/base/9 dcterms:title "Please specify"@en.
http://semweb.ch/base/9 dcterms:description "Please specify"@en.
http://semweb.ch/base/9 dcterms:hasVersion "Please specify".
http://semweb.ch/base/9 dcterms:modified "2016-11-27T13:09:14.332+01:00"^^http://www.w3.org/2001/XMLSchema#dateTime.
http://semweb.ch/base/9 dcterms:identifier "Please specify".
http://semweb.ch/base/9 dc:publisher "Please specify".
http://semweb.ch/base/9 dcterms:rights "Please specify".
http://semweb.ch/base/9 cco:license "Please specify".
http://semweb.ch/base/9 cco:morePermissions "Please specify".
http://semweb.ch/base/9 cco:attributionURL "Please specify".
http://semweb.ch/base/9 cco:attributionName "Please specify".
http://semweb.ch/base/9 foaf:homepage "Please specify".
http://semweb.ch/base/9 foaf:page "Please specify".
http://semweb.ch/base/9 dcterms:issued "Please specify".
http://semweb.ch/base/9 nkos:sizeNote "Please specify".
http://semweb.ch/base/9 nkos:kosType "Please specify".
http://semweb.ch/base/9 nkos:updateFrequency "Please specify".
http://semweb.ch/base/9 void:vocabulary "Please specify".
http://semweb.ch/base/9 void:sparqlEndpoint "Please specify".
http://semweb.ch/base/9 void:rootResource "Please specify".
http://semweb.ch/base/9 void:exampleResource "Please specify".
http://semweb.ch/base/9 void:feature "Please specify".
http://semweb.ch/base/9 owl:versionInfo "Please specify".
}
WHERE {}

Issuing these two SPARQL-Queries on a fresh created Blazegraph namespace results in a correct exploration of the URI's http://semweb.ch/base/9, resp. http://semweb.ch/base/19 showing each a distinct entity, one for each URI.

The "unexpected" thing: In case I load - again on a fresh created namespace with the same options - the well-known FOAF ontology from http://xmlns.com/foaf/spec/20140114.rdf and then issue both (different) insertion queries (see above), the result of exploration of the URI's http://semweb.ch/base/9, resp. http://semweb.ch/base/19 shows an interesting outcome (see below) where both terms are linked via a "sameAs" predicate and the attributes are merged in each of the entities:

Exploring http://semweb.ch/base/9 gives a similar outcome as exploring http://semweb.ch/base/19:

<http://semweb.ch/base/9>

Outgoing Links

rdf:typeowl:Thing
rdf:typeskos:ConceptScheme
owl:sameAs<http://semweb.ch/base/19>

Incoming Links

<http://semweb.ch/base/19>owl:sameAs

Attributes

<http://creativecommons.org/ns#attributionName>Please specify
<http://creativecommons.org/ns#attributionURL>Please specify
<http://creativecommons.org/ns#license>Please specify
<http://creativecommons.org/ns#morePermissions>Please specify
dcterm:created2016-11-27T12:09:14.332Z
dcterm:creatorPlease specify
dcterm:identifierPlease specify
dcterm:rightsPlease specify
dcterm:subjectPlease specify
<http://purl.org/nkos/kosType>Please specify
<http://purl.org/nkos/sizeNote>Please specify
<http://purl.org/nkos/updateFrequency>Please specify
owl:versionInfoPlease specify
foaf:homepagePlease specify
foaf:isPrimaryTopicOfPlease specify
foaf:makerPlease specify
foaf:pagePlease specify
skos:prefLabelCCC9
skos:prefLabelCCC19
dcterm:descriptionPlease specify
dcterm:hasVersionPlease specify
dcterm:issuedPlease specify
dcterm:modified2016-11-27T12:09:14.332Z
dcterm:titlePlease specify
dc:publisherPlease specify
void:exampleResourcePlease specify
void:featurePlease specify
void:rootResourcePlease specify
void:sparqlEndpointPlease specify
void:vocabularyPlease specify

Of course, this interesting result is not what I need here - I would just need both entities separately in the RDF store. This behaviour appears only after having loaded the OWL FOAF ontology.

First question: Please what can I do in order to create but also select both entities separately ? Note that these entities (creation of Concept Schemes) can and should created at any time, before or after having loaded an ontology. Of course up to now I need truth maintenance (otherwise I would have to implement it by myself, this would explode the implementation time).

Second question: Obviously this is one of the (hidden?) behaviours of Blazegraph inferences, which are sketched (just sketched) at https://wiki.blazegraph.com/wiki/index.php/InferenceAndTruthMaintenance (thank you), -> Is there please a white paper or a real explanatory link (or maybe a web page) explaining what exactely is done on which option by the inference engine ?

Thank you in advance
Regards

MULTIPLE SPARQL-UPDATES in one shot

Hello

The following double SPARQL-QUERY

DELETE DATA {<http://semweb.ch/base/2-age-1> <http://www.w3.org/2000/01/rdf-schema#label> "Please specify"@en .} ; 
INSERT DATA{<http://semweb.ch/base/2-age-1> <http://www.w3.org/2000/01/rdf-schema#label> "33"@en .} ; 

Sent as payload in one single REST POST request seems not to perform the DELETE part, inverting the sequence does not bring any difference.

I suppose this is a BUG, since it is allowed to submit several SPARQL-UPDATE queries separated by ";"

Note:
When I insert both SPARQL-UPDATE queries in Blazegraph workbench (cut&paste by hand from the above SPARQL-UPDATE code) both queries are executed !!!!

What can I do in order to have multiple queries get executed via REST ?

Thanks in advance

Blazegraph 2.1.4 debian package - bug in configs and/or in /usr/bin/loadRestAPI.sh

Followed the instructions in the blazegraph-deb section to build and install 2.1.4 on Ubuntu 16.04.

Pertains to blazegraph-deb/src/deb/bin/loadRestAPI.sh

$ /usr/bin/loadRestAPI.sh  /data/rdf/
Loading with properties...
quiet=false
verbose=0
closure=false
durableQueues=true
#Needed for quads
#defaultGraph=
com.bigdata.rdf.store.DataLoader.flush=false
com.bigdata.rdf.store.DataLoader.bufferCapacity=100000
com.bigdata.rdf.store.DataLoader.queueCapacity=10
#Namespace to load
namespace=kb
#Files to load
fileOrDirs=/data/src/dlss/ld4l/ld4p_root/rdf/MarcRDF/

##
## ATT: this is the bug here:
##
#Property file (if creating a new namespace)
propertyFile=/etc/blazegraph/RWStore.properties/RWStore.properties

DATALOADER-SERVLET: kb
java.util.concurrent.ExecutionException: java.io.FileNotFoundException: /etc/blazegraph/RWStore.properties/RWStore.properties
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:188)

When the loadRestAPI.sh script sources the /etc/default/blazegraph it already gets the full path to the RWStore.properties file, i.e.

$ grep 'RWStore' /etc/default/blazegraph
NSS_PROPERTIES="${BLZG_CONF}"/RWStore.properties

$ grep -C1 'NSS_PROPERTIES' /usr/bin/loadRestAPI.sh 

[ -z "${NSS_PROPERTIES}" ] && NSS_PROPERTIES=/etc/blazegraph

export NSS_DATALOAD_PROPERTIES="${NSS_PROPERTIES}/RWStore.properties"

Not using W3C OWL Time ontology time:number correctly?

A colleague and I are seeing incorrect results from Blazegraph when using the W3C OWL Time ontology time:Number RDFS Class (ref: https://www.w3.org/TR/owl-time/#time:Number )

Loading the graph: https://github.com/dr-shorthair/GeologicTimeScale/blob/master/ISC/isc2016-12-time.ttl

and using the SPARQL queries at: dr-shorthair/GeologicTimeScale#1

results in invalid returns for all three using 2.1.1 and 2.1.5 SNAPSHOT. With the query for 66 million years (this a Geologic Time graph used in deep time geological science work) no results return, for the others a subset of valid results returns. We should get results with isc:Phanerozoic as a single result for 66. For 65 and 67 million years Blazegraph returns 3 results, but there should have been 5 in each case.

Simon Cox of CSIRO Astralia notes the results expected and a potential issue around reading time:Numeric in his comment at dr-shorthair/GeologicTimeScale#1 (comment)

This query works in TopBraid, but not in Blazegraph as he notes.

RPM package should use the alternatives settings for JRE

The RPM package for Blazegraph does not run out-of-the-box on RHEL. It seems that the package depends on a manual configuration of JAVA_HOME. For RHEL, the "alternatives" tool is used to set the default JRE. The RPM package should probably use the conventions from alternatives instead of requiring manual setting of the JAVA_HOME as part of the service startup.

Some relevant references might be:

Test Instance:

$ cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.3 (Maipo)

$ sudo service blazegraph start
Starting blazegraph (via systemctl): Job for blazegraph.service failed because the control process exited with error code. See "systemctl status blazegraph.service" and "journalctl -xe" for details.
[FAILED]

Jan 23 09:46:39 u549486-dev systemd: Cannot add dependency job for unit microcode.service, ignoring: Unit is not loaded properly: Invalid argument.
Jan 23 09:46:39 u549486-dev systemd: Starting LSB: Start Blazegraph...
Jan 23 09:46:39 u549486-dev blazegraph: no JDK or JRE found - please set JAVA_HOME
Jan 23 09:46:39 u549486-dev systemd: blazegraph.service: control process exited, code=exited status=1
Jan 23 09:46:39 u549486-dev systemd: Failed to start LSB: Start Blazegraph.
Jan 23 09:46:39 u549486-dev systemd: Unit blazegraph.service entered failed state.
Jan 23 09:46:39 u549486-dev systemd: blazegraph.service failed.

Error when loading a bulk .zip containing database

Hi,
I am trying to use Blazegraph to load a .zip database triples of over 4GB on disk. I am on windows and this is the error I get: No parser factory for available for RDF format null. Is it that .zip compressed files are not accepted by the loader? I am using the command:
java -cp blazegraph.jar com.bigdata.rdf.store.DataLoader myfile.properties d:\path\to\dataset\name.zip

How not to listen to all interfaces

Hello, I'm a beginner with blazegraph.
I downloaded blazegraph(2.1.1).jar
I followed the getting started (a little modified because it otherwise fails without the encoding) :
java -Dfile.encoding=UTF-8 -Dsun.jnu.encoding=UTF-8 -server -Xmx4g -jar blazegraph.jar

The server run fine but it says
serviceURL: http://privateip:9999

but it's also accessible on http://publicip:9999

The server is accessible and fully operational on the public interface (and without password).
How do I configure it so it's not the case ?
Is there a way to configure ssl and password (I searched the doc but could not find it, can you pinpoint it)?
For the moment I can put a filter in iptables but I would rather not have blazegraph listen on the public ip.

Thank you

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.