terminusdb / terminus_store_prolog Goto Github PK
View Code? Open in Web Editor NEWProlog binding for terminusdb-store
License: Apache License 2.0
Prolog binding for terminusdb-store
License: Apache License 2.0
Bitmap indexing was written in a rush, and there's some TODO's in that code regarding how to pick a proper logarray width.
The rust library can return a label list, but this is not yet exposed in the prolog library. Someone should add it.
people will want to store ints and floats and such
Trying the first example in the readme file after successfully installing the pack:
?- use_module(library(terminus_store)).
true.
?- open_directory_store("testdir", Store),
| open_write(Store, Builder),
| open_named_graph(Store, "sometestdb", DB),
| nb_add_triple(Builder, "Subject", "Predicate", value("Object")),
| nb_commit(Builder, Layer),
| nb_set_head(DB, Layer).
ERROR: Unhandled exception: terminus_store_rust_error('No such file or directory (os error 2)')
Tracing the query gives:
Call: (11) terminus_store:open_directory_store("testdir", _5696) ? creep
Exit: (11) terminus_store:open_directory_store("testdir", <store_blob>) ? creep
Call: (11) terminus_store:open_write(<store_blob>, _5702) ? creep
ERROR: Unhandled exception: terminus_store_rust_error('No such file or directory (os error 2)')
Exception: (11) terminus_store:open_write(<store_blob>, _5702) ?
The current scheme with builder and layer and getting a builder is pretty confusing. Perhaps head should be guaranteed to return some layer or something - the open_write the first time on the graph, and from then on on the layer was baffling.
Right now, every parameter to nb_remove_triple/4
needs to be ground. For mass removal, it'd be useful if parameters could be non-ground, meaning any matching triple is to be deleted.
The section Convert strings to ids and query by id
uses old artity 2 predicates for subject_id, predicate_id, and object_id.
It works if changed to:
open_directory_store("testdir", Store),
open_named_graph(Store, "sometestdb", DB),
head(DB, Layer),
subject_id(Layer, "Subject", S_Id),
id_triple(Layer, S_Id, P_Id, O_Id),
predicate_id(Layer, Predicate, P_Id),
object_id(Layer, Object, O_Id).
Trying to install the pack in macOS (running SWI-Prolog 8.1.16) fails with the error:
% Compiling tokio-uds v0.2.5
% Compiling tokio v0.1.22
% Compiling terminus-store v0.7.6
% Compiling terminus-store-prolog v0.1.0 (/Users/pmoura/.local/share/swi-prolog/pack/terminus_store_prolog/rust)
% Finished release [optimized] target(s) in 1m 08s
% cp rust/target/release//libterminus_store_prolog.so lib/x86_64-darwin
% cp: rust/target/release//libterminus_store_prolog.so: No such file or directory
% make: *** [build] Error 1
The error results from the Rust compilation generates a .dylib
file instead of a .so
file:
$ ll /Users/pmoura/.local/share/swi-prolog/pack/terminus_store_prolog/rust/target/release/
total 5904
drwxr-xr-x 22 pmoura staff 704 Nov 22 15:10 build
drwxr-xr-x 160 pmoura staff 5120 Nov 22 15:11 deps
drwxr-xr-x 2 pmoura staff 64 Nov 22 15:10 examples
drwxr-xr-x 2 pmoura staff 64 Nov 22 15:10 incremental
-rw-r--r-- 1 pmoura staff 198 Nov 22 15:11 libterminus_store_prolog.d
-rwxr-xr-x 2 pmoura staff 3017552 Nov 22 15:11 libterminus_store_prolog.dylib
drwxr-xr-x 2 pmoura staff 64 Nov 22 15:10 native
The .so
extension is specified in the `Makefile:
$ cat /Users/pmoura/.local/share/swi-prolog/pack/terminus_store_prolog/Makefile
RUST_LIB_NAME = terminus_store_prolog
RUST_LIB = lib$(RUST_LIB_NAME).$(SOEXT)
RUST_TARGET=release
RUST_TARGET_DIR = rust/target/$(RUST_TARGET)/
SOEXT = so
But the SOEXT
value is taken from the buildenv.sh
file generated by the query prolog_pack:save_build_environment('./')
. I don't know why SWI-Prolog is setting the extension to .so
in macOS where the default for shared libraries is .dylib
. Thus, it's not clear where this issue should be fixed.
We have integrated terminus_store_prolog
into the terminusdb
repository (terminusdb/terminusdb#1160), so we should leave a note on the README.md
and archive the terminus_store_prolog
repository.
Currently we use some bash script wrapper to get the default directories correct so all the shared libs load. This is ugly and incorrect and we should fix this. I started writing a load tester and quickly discovered it's painful.
Anyway, it doesn't seem 'what should be done'.
Trying to install the pack (using the pack_install(terminus_store_prolog)
query on Ubuntu 18.04.3 using cargo
and rustc
1.36.0 fails with the compilation error:
% Compiling terminus-store v0.7.6
ERROR: error: enum variants on type aliases are experimental
ERROR: --> /home/pmoura/.cargo/registry/src/github.com-1ecc6299db9ec823/terminus-store-0.7.6/src/layer/layer.rs:459:13
ERROR: |
ERROR: 459 | Self::Unresolved(_) => false,
ERROR: | ^^^^^^^^^^^^^^^^^^^
It would be convenient to have a create_or_open_named_graph instead of having to somehow know if the graph exists and call either create or open.
The $SWIARCH
test in the Makefile only matches on x86_64-darwin
, so it tries to build using the default (Linux) $BUILD_LD_OPTIONS
. These options aren't understood by the XCode linker, so the build fails.
I tried adding an arm64-darwin
test arm that uses the same $BUILD_LD_OPTIONS
; that build succeeds and passes the tests.
I'm not sure whether the "correct" fix would be to switch on $SWIARCH
or $OS
, though, since I don't know the reasons why the Darwin test is architecture-specific.
SWI-Prolog version 8.4.1 for arm64-darwin
cargo 1.57.0 (b2e52d7ca 2021-10-21)
Also fails to build locally with the same error.
...
% Compiling swipl-macros v0.3.3
% Compiling thiserror v1.0.30
% Compiling futures-executor v0.3.17
% Compiling futures v0.3.17
% Compiling swipl-fli v0.3.2
% Compiling futures-locks v0.6.0
% Compiling tokio-util v0.6.8
% Compiling terminus-store v0.19.5
% Compiling swipl v0.3.5
% Compiling terminus-store-prolog v0.2.0 (/Users/avanti/.local/share/swi-prolog/pack/terminus_store_prolog/rust)
% Finished release [optimized] target(s) in 40.34s
% cp rust/target/release/libterminus_store_prolog.so \
% lib/arm64-darwin/libterminus_store.so
% cp: rust/target/release/libterminus_store_prolog.so: No such file or directory
% make: *** [build] Error 1
ERROR: Process "process(path(make),[all])": exit status: 2
ERROR: In:
ERROR: [21] throw(error(process_error(...,...),_11834))
ERROR: [20] prolog_pack:run_process(path(make),[all],[directory('/Users/avanti/.local/share/swi-prolog/pack/terminus_store_prolog'),...]) at /opt/homebrew/Cellar/swi-prolog/8.4.1/libexec/lib/swipl/library/prolog_pack.pl:2333
ERROR: [19] forall(prolog_pack:member(all,...),prolog_pack:run_process(...,...,...)) at /opt/homebrew/Cellar/swi-prolog/8.4.1/libexec/lib/swipl/boot/apply.pl:52
ERROR: [18] prolog_pack:pack_make('/Users/avanti/.local/share/swi-prolog/pack/terminus_store_prolog',[all,check|...],[inquiry(true),...|...]) at /opt/homebrew/Cellar/swi-prolog/8.4.1/libexec/lib/swipl/library/prolog_pack.pl:1252
ERROR: [17] prolog_pack:make_foreign('/Users/avanti/.local/share/swi-prolog/pack/terminus_store_prolog',[inquiry(true),...|...]) at /opt/homebrew/Cellar/swi-prolog/8.4.1/libexec/lib/swipl/library/prolog_pack.pl:1231
ERROR: [16] prolog_pack:post_install_foreign(terminus_store_prolog,'/Users/avanti/.local/share/swi-prolog/pack/terminus_store_prolog',[inquiry(true),...|...]) at /opt/homebrew/Cellar/swi-prolog/8.4.1/libexec/lib/swipl/library/prolog_pack.pl:1116
ERROR: [15] prolog_pack:pack_post_install(terminus_store_prolog,'/Users/avanti/.local/share/swi-prolog/pack/terminus_store_prolog',[inquiry(true),...|...]) at /opt/homebrew/Cellar/swi-prolog/8.4.1/libexec/lib/swipl/library/prolog_pack.pl:1077
ERROR: [14] prolog_pack:pack_install_from_local('/Users/avanti/.local/share/swi-prolog/pack/Downloads/terminus_store_prolog-0.19.5.zip','/Users/avanti/.local/share/swi-prolog/pack',terminus_store_prolog,[inquiry(true),...|...]) at /opt/homebrew/Cellar/swi-prolog/8.4.1/libexec/lib/swipl/library/prolog_pack.pl:768
ERROR: [13] prolog_pack:pack_install_from_url(https,'https://github.com/terminusdb/terminus_store_prolog/archive/v0.19.5.zip','/Users/avanti/.local/share/swi-prolog/pack',terminus_store_prolog,[inquiry(true),...|...]) at /opt/homebrew/Cellar/swi-prolog/8.4.1/libexec/lib/swipl/library/prolog_pack.pl:995
ERROR: [12] prolog_pack:pack_install(terminus_store_prolog,'/Users/avanti/.local/share/swi-prolog/pack',[inquiry(true),...|...]) at /opt/homebrew/Cellar/swi-prolog/8.4.1/libexec/lib/swipl/library/prolog_pack.pl:738
ERROR: [11] prolog_pack:pack_install(terminus_store_prolog,[pack(terminus_store_prolog),...|...]) at /opt/homebrew/Cellar/swi-prolog/8.4.1/libexec/lib/swipl/library/prolog_pack.pl:653
ERROR: [10] prolog_pack:pack_install(terminus_store_prolog) at /opt/homebrew/Cellar/swi-prolog/8.4.1/libexec/lib/swipl/library/prolog_pack.pl:475
ERROR: [9] toplevel_call(user:user: ...) at /opt/homebrew/Cellar/swi-prolog/8.4.1/libexec/lib/swipl/boot/toplevel.pl:1117
In the definition of the following predicate:
is there any reason to use a proprietary compound term with zero arguments in clause head instead of an atom? All calls to this predicate don't seem to require it.
Found this issue while debugging some odd exceptions that occur with SWI-Prolog 8.0.3 but not with 8.1.16, which suggests that clause head is triggering a SWI-Prolog bug in the stable version that's already fixed in the current devel version.
Label deletion is implemented in the store library, but not yet in the prolog library. Make it so.
We should check for directory existance when opening the store.
On a machine WITHOUT a /tmp/demo directory:
Welcome to SWI-Prolog (threaded, 64 bits, version 8.0.3)
SWI-Prolog comes with ABSOLUTELY NO WARRANTY. This is free software.
Please run ?- license. for legal details.
For online help and background, visit http://www.swi-prolog.org
For built-in help, use ?- help(Topic). or ?- apropos(Word).
?- open_directory_store('/tmp/demo', Store).
Store = <store_blob> .
?- open_write($Store, Builder).
ERROR: Unhandled exception: terminus_store_rust_error('No such file or directory (os error 2)')
EXPECTED: throw exception from open_directory_store/2
OBSERVED: lazily only flips when the store's written
In the current implementation, doing id_triple($Layer, S, P, O)
will call various nondeterministic foreign predicates, all creating blobs:
id_triple(Layer, Subject, Predicate, Object) :-
lookup_subject(Layer, Subject_Lookup),
subject_lookup_subject(Subject_Lookup, Subject),
subject_lookup_predicate(Subject_Lookup, Predicate_Lookup),
subject_predicate_lookup_predicate(Predicate_Lookup, Predicate),
subject_predicate_lookup_object(Predicate_Lookup, Object).
Here, lookup_subject/2
and subject_lookup_predicate/2
are both nondet predicates which bind a lookup blob to their second argument, leaving a choicepoint to bind a different on on backtrack.
This means that just iterating over a dataset will create a lot of lookup blobs (as many as distinct_subjects + distinct_subject_predicate_pairs), all of which are almost instantly forgotten and left for the garbage collector to collect.
Lookups are nice in theory, cause it allows us to implement more complex lookup patterns, where we can cache parts of the lookup. For the basic triple lookup though, they are inefficient. Much better would be for the iteration to happen entirely on the native side, which should not return lookup blobs but rather bind the subject/predicate/object id directly.
Schema checking needs to check to see that comment and label fields are well typed before allowing them to be entered.
swipl run_test.pl -- -n 1
?- tspy(opt_test).
?-go.
Graphic debugger will stop at opt_test(one_big_union_er_layer)
s to step over
EXPECTED: last item on stack is opt_test
OBSERVED: 10_000 choice points for terminus_store:subject_predicate_lookup_object/2
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.