rescrv / hyperdex Goto Github PK
View Code? Open in Web Editor NEWHyperDex is a scalable, searchable key-value store
Home Page: http://hyperdex.org
License: BSD 3-Clause "New" or "Revised" License
HyperDex is a scalable, searchable key-value store
Home Page: http://hyperdex.org
License: BSD 3-Clause "New" or "Revised" License
HyperDex is a distributed, searchable, and consistent key-value store. More information is available at http://hyperdex.org/ Installation instructions available under doc/
Per Robert Escriva's email:
Currently, the system is assumed to be always on. Shutting down the
system without violating the consistency guarantees in the process
requires global coordination to flush outstanding operations. This will
be part of a near-future release of HyperDex as it's an important
feature to have.
There is currently no documentation about the persistence mechanism used for hyperdex. Answers to the following would be a good start:
Hi,
the tutorial on how to create a new space is this example:
$ hyperdex-coordinator-control --host 127.0.0.1 --port 6970 add-space << EOF
space phonebook
dimensions username, first, last, phone (uint64)
key username auto 1 3
subspace first, last, phone auto 2 3
EOF
however I get this error message:
Expected "key" (at char 56), (line:2, col:41)
Creating the space like this:
$ hyperdex-coordinator-control --host 127.0.0.1 --port 6970 add-space << EOF
space phonebook
dimensions username, first, last, phone (int64)
key username auto 1 3
subspace first, last, phone auto 2 3
EOF
works as expected (giving the type as int64 instead of uint64).
OS: OpenSuSE 12.1 x86_64
hyperdex-0.2b9
Cheers
Peter
There is currently only minimal documentation about cluster elasticity (or it is not explicitly marked as such.) A good start would be answers to the following:
Traceback (most recent call last):
File "scripts/client.py", line 78, in
t = hc.benchmarkGet()
File "scripts/client.py", line 59, in benchmarkGet
self.get(self.data[x]['name'])
File "scripts/client.py", line 26, in get
d = self.c.async_get(self.s, f_name)
File "hyperclient.pyx", line 955, in hyperclient.Client.async_get (hyperclient/python/hyperclient.c:16262)
File "hyperclient.pyx", line 509, in hyperclient.DeferredGet.cinit (hyperclient/python/hyperclient.c:7265)
hyperclient.HyperClientException: Unknown Error (file a bug)
When trying to put() any complex type -- list, set or map -- that contain int64 or float elements/key/value primitives I get a Server Error
How to reproduce:
Space definition:
hyperdex-coordinator-control --host 127.0.0.1 --port 6970 add-space << EOF
space phonebook
dimensions username, first, last, pets (map (string, string)), phone (map (string, int64)), address (map (int64, string)), pant (map (int64, int64))
key username auto 1 3
subspace first, last auto 2 3
EOF
Python session where the put() fails:
Python 2.7.3 (default, May 29 2012, 14:54:22)
[GCC 4.6.3 20120306 (Red Hat 4.6.3-2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
import hyperclient
c = hyperclient.Client('127.0.0.1', 1234)
c.put('phonebook', 'jsmith1', {'first': 'John', 'last': 'Smith', 'pets': {'padthai': 'dog', 'pebbles': 'bird', 'felix': 'cat'}, 'phone': {'home': 4165551024, 'work': 4165551025, 'mobile': 4165551026}, 'address': {15: 'Islington Ave', 34: 'West Hoyven St.', 5123: 'Cladwell Cresent'}, 'pant': { 34: 32, 36: 34, 40: -5}})
Traceback (most recent call last):
File "", line 1, in
File "hyperclient.pyx", line 1126, in hyperclient.Client.put (hyperclient/python/hyperclient.c:17487)
File "hyperclient.pyx", line 774, in hyperclient.DeferredFromAttrs.wait (hyperclient/python/hyperclient.c:12213)
hyperclient.HyperClientException: HyperClient(HYPERCLIENT_SERVERERROR, Server Error)
Python session where the put() succeeds: ie., don't put() any complex attrs with int64 or float types
Python 2.7.3 (default, May 29 2012, 14:54:22)
[GCC 4.6.3 20120306 (Red Hat 4.6.3-2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
import hyperclient
c = hyperclient.Client('127.0.0.1', 1234)
c.put('phonebook', 'jsmith1', {'first': 'John', 'last': 'Smith', 'pets': {'padthai': 'dog', 'pebbles': 'bird', 'felix': 'cat'}})
True
I get the exact same exception in java:
Exception in thread "main" hyperclient.HyperClientException: Server Error
at hyperclient.DeferredFromAttrs.waitFor(DeferredFromAttrs.java:57)
at hyperclient.HyperClient.put(HyperClient.java:734)
at HyperPut.main(HyperPut.java:74)
With the code:
import hyperclient.*;
import java.math.*;
import java.util.*;
public class HyperPut
{
public static void main(String[] args) throws Exception
{
String key = args[0];
String first = args[1];
String last = args[2];
//String phone = args[3];
HashMap<String,Object> attributes = new HashMap<String,Object>();
Vector<Object> list = new Vector<Object>();
//list.add("dog");
//list.add("cat");
//list.add("tree");
//list.add("shrub");
//list.add(null);
//list.add(new Long(10));
//list.add(new Long(5));
HashSet<Object> set = new HashSet<Object>();
set.add("tree");
set.add("shrub");
//set.add("bat");
//set.add("cat");
//set.add("dog");
//set.add(null);
//set.add(new Long(10));
//set.add(new Long(5));
HashMap<Object,Object> pets = new HashMap<Object,Object>();
pets.put("padthi","dog");
pets.put("felix","cat");
pets.put("pebbles","bird");
HashMap<Object,Long> phone = new HashMap<Object,Long>();
/*
phone.put("home",new Long(4165551024L));
phone.put("work",new Long(4165551025L));
phone.put("mobile",new Long(4165551026L));
*/
phone.put("home",4165551034L);
phone.put("work",4165551035L);
phone.put("mobile",4165551036L);
HashMap<Object,Object> address = new HashMap<Object,Object>();
address.put(new Long(34),"Wayward Dr.");
address.put(new Long(1024),"Silicon Rd.");
address.put(new Long(25),"Green Strings Crt.");
HashMap<Object,Object> pant = new HashMap<Object,Object>();
pant.put(new Long(34),new Long(32));
pant.put(new Long(36),new Long(30));
pant.put(new Long(40),new Long(52));
attributes.put("first",first);
attributes.put("last",last);
//attributes.put("bogus","bogus");
//attributes.put("phone",Long.parseLong(phone));
//attributes.put("phone",list);
//attributes.put("phone",set);
attributes.put("pets",pets);
attributes.put("phone",phone);
attributes.put("address",address);
attributes.put("pant",pant);
HyperClient c = new HyperClient("127.0.0.1",1234);
System.out.println(c.put("phonebook",key,attributes));
attributes.put("first","George");
System.out.println(c.put("phonebook","gtolomic",attributes));
//Deferred d = c.async_put("phonebook",key,attributes);
//System.out.println(d.waitFor());
}
}
This used to completely work in python and java.
While OSX supports native pthreads, a few missing functions keep libe and libpo6 from building (and sadly keeps HyperDex from compiling on OSX).
Specifically, OSX lacks:
pthread_barriers
pthread_spinlock
Perhaps you could use the more robust primitives from:
www.concurrencykit.org
Also, you are using:
REALTIME_CLOCK which is not supported on OSX.
Use mach_absolute time instead:
http://developer.apple.com/library/mac/#qa/qa1398/_index.html
....
Compile errors occur in benchmarks subfolder:
make[2]: Entering directory .../libe/0.2.7/benchmarks' CXX lockfree_fifo.o CXX lockfree_hash_map.o CXX lockfree_hash_set.o CXX locking_iterable_fifo.o CXX nonblocking_bounded_fifo.o In file included from locking_iterable_fifo.cc:42: ...libpo6/0.2.3/include/po6/threads/barrier.h:57: error: ‘pthread_barrier_t’ does not name a type ...libpo6/0.2.3/include/po6/threads/barrier.h: In constructor ‘po6::threads::barrier::barrier(size_t)’: ...libpo6/0.2.3/include/po6/threads/barrier.h:62: error: class ‘po6::threads::barrier’ does not have any field named ‘m_barrier’ ...libpo6/0.2.3/include/po6/threads/barrier.h:64: error: ‘m_barrier’ was not declared in this scope ...libpo6/0.2.3/include/po6/threads/barrier.h:64: error: ‘pthread_barrier_init’ was not declared in this scope ...libpo6/0.2.3/include/po6/threads/barrier.h: In destructor ‘po6::threads::barrier::~barrier()’: ...libpo6/0.2.3/include/po6/threads/barrier.h:75: error: ‘m_barrier’ was not declared in this scope ...libpo6/0.2.3/include/po6/threads/barrier.h:75: error: ‘pthread_barrier_destroy’ was not declared in this scope ...libpo6/0.2.3/include/po6/threads/barrier.h: In member function ‘bool po6::threads::barrier::wait()’: ...libpo6/0.2.3/include/po6/threads/barrier.h:88: error: ‘m_barrier’ was not declared in this scope ...libpo6/0.2.3/include/po6/threads/barrier.h:88: error: ‘pthread_barrier_wait’ was not declared in this scope ...libpo6/0.2.3/include/po6/threads/barrier.h:90: error: ‘PTHREAD_BARRIER_SERIAL_THREAD’ was not declared in this scope ...libpo6/0.2.3/include/po6/threads/barrier.h:95: error: ‘PTHREAD_BARRIER_SERIAL_THREAD’ was not declared in this scope In file included from locking_iterable_fifo.cc:47: .../libe/0.2.7/include/e/timer.h: In function ‘uint64_t e::time()’: .../libe/0.2.7/include/e/timer.h:50: error: ‘CLOCK_REALTIME’ was not declared in this scope .../libe/0.2.7/include/e/timer.h:50: error: ‘clock_gettime’ was not declared in this scope .../libe/0.2.7/include/e/timer.h: In member function ‘void e::stopwatch::reset()’: .../libe/0.2.7/include/e/timer.h:127: error: ‘CLOCK_REALTIME’ was not declared in this scope .../libe/0.2.7/include/e/timer.h:127: error: ‘clock_gettime’ was not declared in this scope .../libe/0.2.7/include/e/timer.h: In member function ‘uint64_t e::stopwatch::resolution()’: .../libe/0.2.7/include/e/timer.h:137: error: ‘CLOCK_REALTIME’ was not declared in this scope .../libe/0.2.7/include/e/timer.h:137: error: ‘clock_getres’ was not declared in this scope .../libe/0.2.7/include/e/timer.h: In member function ‘uint64_t e::stopwatch::peek()’: .../libe/0.2.7/include/e/timer.h:149: error: ‘CLOCK_REALTIME’ was not declared in this scope .../libe/0.2.7/include/e/timer.h:149: error: ‘clock_gettime’ was not declared in this scope In file included from .../libe/0.2.7/include/e/locking_iterable_fifo.h:40, from locking_iterable_fifo.cc:48: ...libpo6/0.2.3/include/po6/threads/spinlock.h: At global scope: ...libpo6/0.2.3/include/po6/threads/spinlock.h:62: error: ‘pthread_spinlock_t’ does not name a type ...libpo6/0.2.3/include/po6/threads/spinlock.h: In constructor ‘po6::threads::spinlock::spinlock()’: ...libpo6/0.2.3/include/po6/threads/spinlock.h:80: error: class ‘po6::threads::spinlock’ does not have any field named ‘m_spin’ ...libpo6/0.2.3/include/po6/threads/spinlock.h:82: error: ‘m_spin’ was not declared in this scope ...libpo6/0.2.3/include/po6/threads/spinlock.h:82: error: ‘pthread_spin_init’ was not declared in this scope ...libpo6/0.2.3/include/po6/threads/spinlock.h: In destructor ‘po6::threads::spinlock::~spinlock()’: ...libpo6/0.2.3/include/po6/threads/spinlock.h:93: error: ‘m_spin’ was not declared in this scope ...libpo6/0.2.3/include/po6/threads/spinlock.h:93: error: ‘pthread_spin_destroy’ was not declared in this scope ...libpo6/0.2.3/include/po6/threads/spinlock.h: In member function ‘void po6::threads::spinlock::lock()’: ...libpo6/0.2.3/include/po6/threads/spinlock.h:106: error: ‘m_spin’ was not declared in this scope ...libpo6/0.2.3/include/po6/threads/spinlock.h:106: error: ‘pthread_spin_lock’ was not declared in this scope ...libpo6/0.2.3/include/po6/threads/spinlock.h: In member function ‘bool po6::threads::spinlock::trylock()’: ...libpo6/0.2.3/include/po6/threads/spinlock.h:117: error: ‘m_spin’ was not declared in this scope ...libpo6/0.2.3/include/po6/threads/spinlock.h:117: error: ‘pthread_spin_trylock’ was not declared in this scope ...libpo6/0.2.3/include/po6/threads/spinlock.h: In member function ‘void po6::threads::spinlock::unlock()’: ...libpo6/0.2.3/include/po6/threads/spinlock.h:136: error: ‘m_spin’ was not declared in this scope ...libpo6/0.2.3/include/po6/threads/spinlock.h:136: error: ‘pthread_spin_unlock’ was not declared in this scope lockfree_hash_map.cc: In function ‘void worker_thread(e::lockfree_hash_map<long long unsigned int, long long unsigned int, id>*)’: lockfree_hash_map.cc:143: error: ‘drand48_data’ was not declared in this scope lockfree_hash_map.cc:143: error: expected
;' before ‘buf’
lockfree_hash_map.cc:147: error: invalid conversion from ‘opaque_pthread_t’ to ‘short unsigned int’
lockfree_hash_map.cc:148: error: ‘buf’ was not declared in this scope
lockfree_hash_map.cc:148: error: ‘seed48_r’ was not declared in this scope
lockfree_hash_map.cc:154: error: ‘lrand48_r’ was not declared in this scope
lockfree_hash_map.cc:163: error: ISO C++ forbids comparison between pointer and integer
lockfree_hash_map.cc:175: error: invalid conversion from ‘opaque_pthread_t’ to ‘long long unsigned int’
lockfree_hash_map.cc:175: error: initializing argument 2 of ‘bool e::lockfree_hash_map<K, V, H>::insert(const K&, const V&) [with K = long long unsigned int, V = long long unsigned int, uint64_t (* H)(const K&) = id]’
lockfree_hash_set.cc: In function ‘void worker_thread(e::lockfree_hash_set<long long unsigned int, id>_)’:
lockfree_hash_set.cc:142: error: ‘drand48_data’ was not declared in this scope
lockfree_hash_set.cc:142: error: expected ;' before ‘buf’ lockfree_hash_set.cc:146: error: invalid conversion from ‘_opaque_pthread_t_’ to ‘short unsigned int’ lockfree_hash_set.cc:147: error: ‘buf’ was not declared in this scope lockfree_hash_set.cc:147: error: ‘seed48_r’ was not declared in this scope lockfree_hash_set.cc:152: error: ‘lrand48_r’ was not declared in this scope make[2]: **\* [locking_iterable_fifo.o] Error 1 make[2]: **\* Waiting for unfinished jobs.... make[2]: **\* [lockfree_hash_set.o] Error 1 make[2]: **\* [lockfree_hash_map.o] Error 1 make[2]: Leaving directory
.../libe/0.2.7/benchmarks'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory`.../libe/0.2.7'
make: *** [all] Error 2
Hey,
I can't get the latest repo to compile because it's looking for a (seemingly non-existent) include in the po6
library.
/hyperdex/hyperdex/coordinatorlink.h:39:30: fatal error: po6/net/hostname.h: No such file or directory
It looks like this was changed recently and my guess is that I will be able to pull behind that commit and get past this issue, but I figured I would let you guys know. I looked at the po6 source as well (here on github), but I can't see any reference to this file there.
Cheers,
Mike
I think I have already success installed the HyperDex.
When I run order "hyperdex-coordinator --control-port 6970 --host-port 1234" as tutorial wrote. It ok.
but When I run the order :"hyperdex-daemon --host 127.0.0.1 --port 1234 --bind-to 127.0.0.2 ", an error happened, it said
"
hyperdex-daemon: error while loading shared libraries: libhyperdisk.so.0: cannot open shared object file: No such file or directory
"
Need Help. Thank You.
Currently the search command allows the predicate of an attribute to be a range of integers whereas the condput command only allows an equality match. There are scenarios where it would be useful to test if the value of an attribute is in a particular interval before updating the object.
Same for every atomic_*
function.
As hyperdex can now store signed int64_t types, shouldn't range predicate types for upper and lower members also be int64_t instead of uint64_t?
Hi,
I built this space:
$ hyperdex-coordinator-control --host 127.0.0.1 --port 6970 add-space << EOF
space test
dimensions id (int64), name
key id auto 1 3
EOF
When trying to insert a row I get:
c.put ('test',1, {'name': 'abc' })
Traceback (most recent call last):
File "", line 1, in
File "hyperclient.pyx", line 827, in hyperclient.Client.put (hyperclient/python/hyperclient.c:11838)
TypeError: Argument 'key' has incorrect type (expected str, got int)
It looks like a key cannot be an integer?
Cheers
Peter
Here's the trivial patch as written by me for gentoo. Feel free to use or improve.
diff -Naur libe-0.2.7/benchmarks/lockfree_hash_map.cc libe-0.2.7-new/benchmarks/lockfree_hash_map.cc
--- libe-0.2.7/benchmarks/lockfree_hash_map.cc 2011-10-12 04:02:24.000000000 +0200
+++ libe-0.2.7-new/benchmarks/lockfree_hash_map.cc 2012-08-20 14:59:39.135789923 +0200
@@ -30,6 +30,7 @@
// C includes
#include
#include <stdint.h>
+#include <unistd.h>
// C++
#include
diff -Naur libe-0.2.7/benchmarks/lockfree_hash_set.cc libe-0.2.7-new/benchmarks/lockfree_hash_set.cc
--- libe-0.2.7/benchmarks/lockfree_hash_set.cc 2011-08-26 16:24:54.000000000 +0200
+++ libe-0.2.7-new/benchmarks/lockfree_hash_set.cc 2012-08-20 15:00:37.877793998 +0200
@@ -30,6 +30,7 @@
// C includes
#include
#include <stdint.h>
+#include <unistd.h>
// C++
#include
Hi,Robert:
I've just successfully compiled hyperdex-0.2b8 in my virtualbox of ubuntu11.10.
then i started the Coordinator, Daemon and Created a phonebook space as the tutorial said, that all goes fine.
but when i did the client task:"import hyperclient", it failed with:
"Traceback (most recent call last):
File "", line 1, in
ImportError: No module named hyperclient "
i'm not quite familiar with python, could you please tell me how can i solve this problem?
thanks.
Here are some informations:
echo $PYTHONPATH
/home/tiger/Share/hyperdex/hyperdex-install/hyperdex/lib/python2.7/site-packages:/home/tiger/Share/hyperdex/hyperdex-install/hyperdex/lib/python2.7/site-packages/hypercoordinator:/home/tiger/Share/hyperdex/hyperdex-install/hyperdex/lib
ls -l /home/tiger/Share/hyperdex/hyperdex-install/hyperdex/lib/python2.7/site-packages/hypercoordinator
total 174
-rwxrwxrwx 1 root root 3992 2012-03-17 22:00 client.py*
-rwxrwxrwx 1 root root 3494 2012-03-17 22:00 client.pyc*
-rwxrwxrwx 1 root root 3494 2012-03-17 22:00 client.pyo*
-rwxrwxrwx 1 root root 27438 2012-03-17 22:00 coordinator.py*
-rwxrwxrwx 1 root root 27509 2012-03-17 22:00 coordinator.pyc*
-rwxrwxrwx 1 root root 27509 2012-03-17 22:00 coordinator.pyo*
-rwxrwxrwx 1 root root 7415 2012-03-17 22:00 hdtypes.py*
-rwxrwxrwx 1 root root 12828 2012-03-17 22:00 hdtypes.pyc*
-rwxrwxrwx 1 root root 12728 2012-03-17 22:00 hdtypes.pyo*
-rwxrwxrwx 1 root root 1560 2012-03-17 22:00 init.py*
-rwxrwxrwx 1 root root 196 2012-03-17 22:00 init.pyc*
-rwxrwxrwx 1 root root 196 2012-03-17 22:00 init.pyo*
-rwxrwxrwx 1 root root 14677 2012-03-17 22:00 parser.py*
-rwxrwxrwx 1 root root 15554 2012-03-17 22:00 parser.pyc*
-rwxrwxrwx 1 root root 15419 2012-03-17 22:00 parser.pyo*
Apologies if some of my questions are answered somewhere, I couldn't find it.
It would be great if the narrative documentation could be supplemented with a structured data-type/command-reference (similar to the redis-reference) so that questions like the above can be easily looked up.
One section in tutorial on wiki does not work. The section is concerning creating a new table. Upon entering nc cmd and subsequent lines. No 'SUCCESS' response is received by nc from HyperDex.
Please advices how to proceed or how to diagnose the problem.
When working with hyperdex, due to it's speed, one of the time costs is requests. Ideally there would be a mechanism to send multiple operations to hyperdex in a single request to minimise this networking cost. Clients may be intelligent enough to split up a multiple request if various indexes are best sent to different nodes within a cluster.
I download the libpo6-0.2.2.tar.gz from http://hyperdex.org/src/, and then my code are:
tar zxvf libpo6-0.2.2.tar.gz
cd libpo6-0.2.2
./configure
make
make install
then I get the following:
Making install in include
make[1]: Entering directory /root/hyperdex/libpo6-0.2.2/include' make[2]: Entering directory
/root/hyperdex/libpo6-0.2.2/include'
make[2]: Nothing to be done for install-exec-am'. test -z "/usr/local/include" || /bin/mkdir -p "/usr/local/include" /bin/mkdir -p '/usr/local/include/po6/io' /usr/bin/install -c -m 644 po6/io/fd.h '/usr/local/include/po6/io' /bin/mkdir -p '/usr/local/include/po6/threads' /usr/bin/install -c -m 644 po6/threads/rwlock.h po6/threads/cond.h po6/threads/spinlock.h po6/threads/thread.h po6/threads/mutex.h po6/threads/barrier.h '/usr/local/include/po6/threads' /bin/mkdir -p '/usr/local/include/po6' /usr/bin/install -c -m 644 po6/error.h po6/noncopyable.h po6/pathname.h '/usr/local/include/po6' /bin/mkdir -p '/usr/local/include/po6/net' /usr/bin/install -c -m 644 po6/net/ipaddr.h po6/net/location.h po6/net/socket.h '/usr/local/include/po6/net' make[2]: Leaving directory
/root/hyperdex/libpo6-0.2.2/include'
make[1]: Leaving directory /root/hyperdex/libpo6-0.2.2/include' Making install in test make[1]: Entering directory
/root/hyperdex/libpo6-0.2.2/test'
make[2]: Entering directory /root/hyperdex/libpo6-0.2.2/test' make[2]: Nothing to be done for
install-exec-am'.
make[2]: Nothing to be done for install-data-am'. make[2]: Leaving directory
/root/hyperdex/libpo6-0.2.2/test'
make[1]: Leaving directory /root/hyperdex/libpo6-0.2.2/test' make[1]: Entering directory
/root/hyperdex/libpo6-0.2.2'
make[2]: Entering directory /root/hyperdex/libpo6-0.2.2' make[2]: Nothing to be done for
install-exec-am'.
test -z "/usr/local/lib/pkgconfig" || /bin/mkdir -p "/usr/local/lib/pkgconfig"
/usr/bin/install -c -m 644 libpo6.pc '/usr/local/lib/pkgconfig'
make[2]: Leaving directory /root/hyperdex/libpo6-0.2.2' make[1]: Leaving directory
/root/hyperdex/libpo6-0.2.2'
But when I configure the "libe-0.2.5" , it shows that :
"
checking for PO6... no
configure: error: Package requirements (libpo6 >= 0.2) were not met:
No package 'libpo6' found
"
Why??? I use RedHat x86_64.
Need Help~~
Thank You.
Hi!
I tried to build hyperdex from source, but didn't find vis.h. On ubuntu, you need to install libbsd-dev to have this file. Imo, you should include it as a configure dependency.
The tests are binary-test, count-test, and replication-stress-test. I tried to run them based on my understanding, e.g. adding spaces with some specific descriptions, and invoking the command line. But nothing happened.
It'd be good if there are instructions on how to run them so once we changed something we have a way to detect regression (of course, more tests are wanted)
Hi,
Can I limitting memory usage of a hyperdex node? Can't find how to do this anywhere.
Is it possible to limit it size into 30MB (like in hypertable)?
Hi,
"
I'm running Ubuntu 12.04. Could you please let me know how to fix this?
Thanks!
Lin
I am testing a sorted_search() that requests the results be sorted by an integer attribute with limit=1 and maximize=true.
The first call to loop() returns with status SUCCESS, and I'm able to read the attributes for the expected record.
The next call to loop(), which I expect to return status SEARCHDONE, instead returns SUCCESS. The attributes remain unchanged, pointing to the first expected record.
A third call to loop() returns -1, with status NONEPENDING.
Test case:
#include <cassert>
#include <iostream>
#include <hyperclient.h>
namespace {
/* space test
* dimensions name, value (int64)
* key name auto 1 3 */
const char *SPACE_TEST = "test";
const char *ATTR_NAME = "name";
const char *ATTR_VALUE = "value";
struct test_record {
std::string name;
int64_t value;
};
std::ostream& operator<<(std::ostream &stream, const test_record &entry)
{
return stream << "{ name: \"" << entry.name
<< "\", value: " << entry.value << " }";
}
int insert(hyperclient *client, const test_record &entry)
{
enum hyperclient_returncode status = HYPERCLIENT_SUCCESS;
const struct hyperclient_attribute attrs[] = {
{
ATTR_VALUE,
(const char*)&entry.value,
sizeof(entry.value),
HYPERDATATYPE_INT64
}
};
const size_t attrs_sz = sizeof(attrs)/sizeof(*attrs);
const int64_t operation = client->put(SPACE_TEST,
entry.name.c_str(),
entry.name.size(),
attrs, attrs_sz,
&status);
std::cout << "client.put(" << entry.name << "=" << entry.value <<
") returned " << operation << " with status " << status << std::endl;
if (operation < 0)
return 0;
const int64_t loop = client->loop(10, &status);
std::cout << "client.loop() returned " << loop
<< " with status " << status << std::endl;
return loop == operation && status == HYPERCLIENT_SUCCESS;
}
int search(hyperclient *client, test_record *result)
{
enum hyperclient_returncode status = HYPERCLIENT_SUCCESS;
struct hyperclient_attribute *attrs;
size_t attrs_sz;
const int64_t operation = client->sorted_search(SPACE_TEST,
NULL, 0,
NULL, 0,
ATTR_VALUE, 1, true,
&status,
&attrs, &attrs_sz);
std::cout << "client.sorted_search() returned "
<< operation << " with status " << status << std::endl;
if (operation < 0)
return 0;
// see if there are any results
int64_t loop = client->loop(10, &status);
std::cout << "client.loop(1) returned " << loop
<< " with status " << status << std::endl;
if (loop != operation)
return 0;
// if there were no results, don't call _destroy_attrs()
if (status == HYPERCLIENT_SEARCHDONE)
return 0;
// read the attributes
for (size_t i = 0; i < attrs_sz; i++)
{
if (strcmp(attrs[i].attr, ATTR_NAME) == 0)
result->name.assign(attrs[i].value,
attrs[i].value + attrs[i].value_sz);
else if (strcmp(attrs[i].attr, ATTR_VALUE) == 0)
result->value = *(int64_t*)(attrs[i].value);
}
std::cout << "result 1: " << *result << std::endl;
// loop again to get SEARCHDONE
loop = client->loop(10, &status);
std::cout << "client.loop(2) returned " << loop
<< " with status " << status << std::endl;
if (status == HYPERCLIENT_SUCCESS)
{
// expected SEARCHDONE.. read the attributes again
test_record result2;
for (size_t i = 0; i < attrs_sz; i++)
{
if (strcmp(attrs[i].attr, ATTR_NAME) == 0)
result2.name.assign(attrs[i].value,
attrs[i].value + attrs[i].value_sz);
else if (strcmp(attrs[i].attr, ATTR_VALUE) == 0)
result2.value = *(int64_t*)(attrs[i].value);
}
std::cout << "result 2: " << result2 << std::endl;
// call loop again for status
loop = client->loop(10, &status);
std::cout << "client.loop(3) returned " << loop
<< " with status " << status << std::endl;
}
// free attribute results
hyperclient_destroy_attrs(attrs, attrs_sz);
return loop == operation && status == HYPERCLIENT_SEARCHDONE;
}
} // namespace
int main(int argc, char *argv[])
{
const test_record a = { "a", 0 };
const test_record b = { "b", 1 };
const test_record c = { "c", 2 };
const test_record d = { "d", 3 };
test_record result;
hyperclient *client;
assert(client = hyperclient_create("127.0.0.1", 1234));
assert(insert(client, a));
assert(insert(client, b));
assert(insert(client, c));
assert(insert(client, d));
assert(search(client, &result));
assert(result.name == "d");
return 0;
}
Produces the output:
client.put(a=0) returned 1 with status HYPERCLIENT_SUCCESS
client.loop() returned 1 with status HYPERCLIENT_SUCCESS
client.put(b=1) returned 2 with status HYPERCLIENT_SUCCESS
client.loop() returned 2 with status HYPERCLIENT_SUCCESS
client.put(c=2) returned 3 with status HYPERCLIENT_SUCCESS
client.loop() returned 3 with status HYPERCLIENT_SUCCESS
client.put(d=3) returned 4 with status HYPERCLIENT_SUCCESS
client.loop() returned 4 with status HYPERCLIENT_SUCCESS
client.sorted_search() returned 5 with status HYPERCLIENT_SUCCESS
client.loop(1) returned 5 with status HYPERCLIENT_SUCCESS
result 1: { name: "d", value: 3 }
client.loop(2) returned 5 with status HYPERCLIENT_SUCCESS
result 2: { name: "d", value: 3 }
client.loop(3) returned -1 with status HYPERCLIENT_NONEPENDING
sorted_search: test_sorted_search.cc:147: int main(int, char**): Assertion `search(client, &result)' failed.
I had to make the following same modification to each file below to make them build:
- .. include:: shards/linearizable.rst
I don't know the syntax of *.rst files, but the space after the '::' struck me as being odd, so I removed it and then everything built
doc/api/shards/atomic_add.rst
doc/api/shards/atomic_and.rst
doc/api/shards/atomic_div.rst
doc/api/shards/atomic_mod.rst
doc/api/shards/atomic_mul.rst
doc/api/shards/atomic_or.rst
doc/api/shards/atomic_sub.rst
doc/api/shards/atomic_xor.rst
doc/api/shards/del.rst
doc/api/shards/get.rst
doc/api/shards/list_lpush.rst
doc/api/shards/list_rpush.rst
doc/api/shards/put.rst
doc/api/shards/set_add.rst
doc/api/shards/set_intersect.rst
doc/api/shards/set_remove.rst
doc/api/shards/set_union.rst
doc/api/shards/string_append.rst
doc/api/shards/string_prepend.rst
Hi, I'm using Lubuntu 12.04... I compiled hyperdex 0.2b9 and all dependencies from source, making sure to use ./configure --enable-python-bindings
. But when I try import hyperclient
in Python, I get the following message: ImportError: libhyperclient.so.0: cannot open shared object file: No such file or directory
.
foobar baz
Hello.
I am trying to put together a simple demo application using hyperdex C api. As a guidance I am using code from python hyperclient. You can see my application at https://github.com/agiz/hyperdex-c-test/blob/master/test_hyperdex.c. What it does is initialize a hyperclient instance and tries to create attributes of an object.
Before running this applications I followed instructions on how to create space described in http://hyperdex.org/doc/tutorial/. I do not know if that is important or not but when creating a space using hyperdex-coordinator-control --host 127.0.0.1 --port 6970 add-space << EOF
and after typing EOF the program does not exits and just stalls. After an hour or two I have interrupted this program and received following output:
^CTraceback (most recent call last):
File "/usr/lib64/python2.6/runpy.py", line 122, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "/usr/lib64/python2.6/runpy.py", line 34, in _run_code
exec code in run_globals
File "/usr/lib/python2.6/site-packages/hypercoordinator/client.py", line 163, in <module>
sys.exit(main(sys.argv[1:]))
File "/usr/lib/python2.6/site-packages/hypercoordinator/client.py", line 158, in main
return args.func(args)
File "/usr/lib/python2.6/site-packages/hypercoordinator/client.py", line 83, in add_space
return send_msg(args.host, args.port, 'add-space', data)
File "/usr/lib/python2.6/site-packages/hypercoordinator/client.py", line 46, in send_msg
d = s.recv(4096)
KeyboardInterrupt
I do however think that space does get created nonetheless.
This is reported by hyperdex-coordinator when creating the space:
INFO:root:Coordinator started
INFO:root:new host uses ID Unidentified(127.0.0.1, 35361)
INFO:root:Unidentified(127.0.0.1, 35361) identified by Instance(addr='127.0.0.2', inport=43296, inver=1, outport=35622, outver=1)
DEBUG:root:Instance(addr='127.0.0.2', inport=43296, inver=1, outport=35622, outver=1) acked config 0
INFO:root:created new space "phonebook"
DEBUG:root:Instance(addr='127.0.0.2', inport=43296, inver=1, outport=35622, outver=1) acked config 1
When I run my test application, test_hyperdex
, status reports 8520 -- HYPERCLIENT_UNKNOWNATTR
.
This is reported by hyperdex-coordinator when I run my program.
INFO:root:new host uses ID Unidentified(127.0.0.1, 35467)
INFO:root:Unidentified(127.0.0.1, 35467) identified by Client(127.0.0.1, 35467)
I am using CentOS 6.2, kernel 2.6.32-220.13.1.el6.x86_64. I have built libe, libpo6 and HyperDex using github sources with gcc version 4.4.6 20110731 (Red Hat 4.4.6-3) (GCC). Before running autoreconf -i
I had to set export LC_ALL=C
and change AC_PREREQ([2.66])
to AC_PREREQ([2.63])
. When running autoreconf -i
in HyperDex for the first time I received:
configure.ac:27: error: possibly undefined macro: AC_PROG_JAR
If this token and others are legitimate, please use m4_pattern_allow.
See the Autoconf documentation.
configure.ac:28: error: possibly undefined macro: AC_PROG_JAVAC
configure.ac:29: error: possibly undefined macro: AC_JNI_INCLUDE_DIR
autoreconf: /usr/bin/autoconf failed with exit status: 1
but running it again reported no errors. I hope these errors are meaningless since I did not use any java bindings.
If there is anything more you need to replicate/solve this issue please let me know.
Regards, Ziga
CXX hyperdaemon/hyperdex_daemon-searches.o
hyperdaemon/searches.cc: In member function ‘void hyperdaemon::searches::group_keyop(const hyperdex::entityid&, const hyperdex::entityid&, uint64_t, const hyperspacehashing::search&, hyperdex::network_msgtype, const e::slice&)’:
hyperdaemon/searches.cc:266:19: error: ‘class e::buffer::packer’ has no member named ‘copy’
In file included from /opt/hyperdex/include/e/lockfree_hash_map.h:36:0,
from ./hyperdisk/hyperdisk/disk.h:43,
from hyperdaemon/searches.cc:32:
Please note that if sequence is different: daemon -> node -> add space, all is fine:
ploj@hive:~/github/HyperDex$ hyperdex-coordinator --control-port 6970 --host-port 1234 --logging debug
INFO:root:Fresh cluster starting.
INFO:root:new host uses ID Unidentified(127.0.0.1, 42736)
INFO:root:Unidentified(127.0.0.1, 42736) identified by Instance(addr='127.0.0.2', inport=60824, inver=1, outport=34956, outver=1)
DEBUG:root:Instance(addr='127.0.0.2', inport=60824, inver=1, outport=34956, outver=1) acked config 0
INFO:root:created new space "phonebook3"
DEBUG:root:Instance(addr='127.0.0.2', inport=60824, inver=1, outport=34956, outver=1) acked config 1
$ autoreconf -i -v;
autoreconf2.50: Entering directory .' autoreconf2.50: configure.ac: not using Gettext autoreconf2.50: running: aclocal -I m4 ${ACLOCAL_FLAGS} autoreconf2.50: configure.ac: tracing autoreconf2.50: running: libtoolize --copy libtoolize: putting auxiliary files in
.'.
libtoolize: copying file ./ltmain.sh' libtoolize: putting macros in AC_CONFIG_MACRO_DIR,
m4'.
libtoolize: copying file m4/libtool.m4' libtoolize: copying file
m4/ltoptions.m4'
libtoolize: copying file m4/ltsugar.m4' libtoolize: copying file
m4/ltversion.m4'
libtoolize: copying file `m4/lt~obsolete.m4'
autoreconf2.50: running: /usr/bin/autoconf
configure.ac:31: error: possibly undefined macro: AC_PROG_JAR
If this token and others are legitimate, please use m4_pattern_allow.
See the Autoconf documentation.
configure.ac:32: error: possibly undefined macro: AC_PROG_JAVAC
configure.ac:33: error: possibly undefined macro: AC_JNI_INCLUDE_DIR
configure.ac:151: error: possibly undefined macro: AC_PYTHON_DEVEL
autoreconf2.50: /usr/bin/autoconf failed with exit status: 1
[callen@sorna HyperDex]$ /usr/local/Cellar/autoconf/2.68/bin/autoreconf -i
configure.ac:10: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:198: AC_LANG_CONFTEST is expanded from...
../../lib/autoconf/general.m4:2671: _AC_LINK_IFELSE is expanded from...
../../lib/autoconf/general.m4:2681: AC_LINK_IFELSE is expanded from...
/usr/bin/../share/aclocal/libtool.m4:1100: _LT_SYS_MODULE_PATH_AIX is expanded from...
/usr/bin/../share/aclocal/libtool.m4:5331: _LT_LINKER_SHLIBS is expanded from...
/usr/bin/../share/aclocal/libtool.m4:5416: _LT_LANG_C_CONFIG is expanded from...
/usr/bin/../share/aclocal/libtool.m4:240: _LT_SETUP is expanded from...
/usr/bin/../share/aclocal/libtool.m4:104: LT_INIT is expanded from...
configure.ac:10: the top level
configure.ac:10: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:198: AC_LANG_CONFTEST is expanded from...
../../lib/autoconf/general.m4:2671: _AC_LINK_IFELSE is expanded from...
../../lib/autoconf/general.m4:2681: AC_LINK_IFELSE is expanded from...
/usr/bin/../share/aclocal/libtool.m4:5331: _LT_LINKER_SHLIBS is expanded from...
/usr/bin/../share/aclocal/libtool.m4:5416: _LT_LANG_C_CONFIG is expanded from...
/usr/bin/../share/aclocal/libtool.m4:240: _LT_SETUP is expanded from...
/usr/bin/../share/aclocal/libtool.m4:104: LT_INIT is expanded from...
configure.ac:10: the top level
configure.ac:16: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:198: AC_LANG_CONFTEST is expanded from...
../../lib/autoconf/general.m4:2671: _AC_LINK_IFELSE is expanded from...
../../lib/autoconf/general.m4:2681: AC_LINK_IFELSE is expanded from...
/usr/bin/../share/aclocal/libtool.m4:1100: _LT_SYS_MODULE_PATH_AIX is expanded from...
/usr/bin/../share/aclocal/libtool.m4:6443: _LT_LANG_CXX_CONFIG is expanded from...
/usr/bin/../share/aclocal/libtool.m4:822: _LT_LANG is expanded from...
/usr/bin/../share/aclocal/libtool.m4:811: LT_LANG is expanded from...
configure.ac:16: the top level
configure.ac:10: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:198: AC_LANG_CONFTEST is expanded from...
../../lib/autoconf/general.m4:2671: _AC_LINK_IFELSE is expanded from...
../../lib/autoconf/general.m4:2681: AC_LINK_IFELSE is expanded from...
aclocal.m4:1112: _LT_SYS_MODULE_PATH_AIX is expanded from...
aclocal.m4:5343: _LT_LINKER_SHLIBS is expanded from...
aclocal.m4:5424: _LT_LANG_C_CONFIG is expanded from...
aclocal.m4:258: _LT_SETUP is expanded from...
aclocal.m4:122: LT_INIT is expanded from...
configure.ac:10: the top level
configure.ac:10: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:198: AC_LANG_CONFTEST is expanded from...
../../lib/autoconf/general.m4:2671: _AC_LINK_IFELSE is expanded from...
../../lib/autoconf/general.m4:2681: AC_LINK_IFELSE is expanded from...
aclocal.m4:5343: _LT_LINKER_SHLIBS is expanded from...
aclocal.m4:5424: _LT_LANG_C_CONFIG is expanded from...
aclocal.m4:258: _LT_SETUP is expanded from...
aclocal.m4:122: LT_INIT is expanded from...
configure.ac:10: the top level
configure.ac:16: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:198: AC_LANG_CONFTEST is expanded from...
../../lib/autoconf/general.m4:2671: _AC_LINK_IFELSE is expanded from...
../../lib/autoconf/general.m4:2681: AC_LINK_IFELSE is expanded from...
aclocal.m4:1112: _LT_SYS_MODULE_PATH_AIX is expanded from...
aclocal.m4:6451: _LT_LANG_CXX_CONFIG is expanded from...
aclocal.m4:834: _LT_LANG is expanded from...
aclocal.m4:823: LT_LANG is expanded from...
configure.ac:16: the top level
glibtoolize: putting auxiliary files in .'. glibtoolize: copying file
./ltmain.sh'
glibtoolize: putting macros in AC_CONFIG_MACRO_DIR, m4'. glibtoolize: copying file
m4/libtool.m4'
glibtoolize: copying file m4/ltoptions.m4' glibtoolize: copying file
m4/ltsugar.m4'
glibtoolize: copying file m4/ltversion.m4' glibtoolize: copying file
m4/lt~obsolete.m4'
configure.ac:10: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:198: AC_LANG_CONFTEST is expanded from...
../../lib/autoconf/general.m4:2671: _AC_LINK_IFELSE is expanded from...
../../lib/autoconf/general.m4:2681: AC_LINK_IFELSE is expanded from...
m4/libtool.m4:1100: _LT_SYS_MODULE_PATH_AIX is expanded from...
m4/libtool.m4:5331: _LT_LINKER_SHLIBS is expanded from...
m4/libtool.m4:5416: _LT_LANG_C_CONFIG is expanded from...
m4/libtool.m4:240: _LT_SETUP is expanded from...
m4/libtool.m4:104: LT_INIT is expanded from...
configure.ac:10: the top level
configure.ac:10: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:198: AC_LANG_CONFTEST is expanded from...
../../lib/autoconf/general.m4:2671: _AC_LINK_IFELSE is expanded from...
../../lib/autoconf/general.m4:2681: AC_LINK_IFELSE is expanded from...
m4/libtool.m4:5331: _LT_LINKER_SHLIBS is expanded from...
m4/libtool.m4:5416: _LT_LANG_C_CONFIG is expanded from...
m4/libtool.m4:240: _LT_SETUP is expanded from...
m4/libtool.m4:104: LT_INIT is expanded from...
configure.ac:10: the top level
configure.ac:16: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:198: AC_LANG_CONFTEST is expanded from...
../../lib/autoconf/general.m4:2671: _AC_LINK_IFELSE is expanded from...
../../lib/autoconf/general.m4:2681: AC_LINK_IFELSE is expanded from...
m4/libtool.m4:1100: _LT_SYS_MODULE_PATH_AIX is expanded from...
m4/libtool.m4:6443: _LT_LANG_CXX_CONFIG is expanded from...
m4/libtool.m4:822: _LT_LANG is expanded from...
m4/libtool.m4:811: LT_LANG is expanded from...
configure.ac:16: the top level
configure.ac:10: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:198: AC_LANG_CONFTEST is expanded from...
../../lib/autoconf/general.m4:2671: _AC_LINK_IFELSE is expanded from...
../../lib/autoconf/general.m4:2681: AC_LINK_IFELSE is expanded from...
m4/libtool.m4:1100: _LT_SYS_MODULE_PATH_AIX is expanded from...
m4/libtool.m4:5331: _LT_LINKER_SHLIBS is expanded from...
m4/libtool.m4:5416: _LT_LANG_C_CONFIG is expanded from...
m4/libtool.m4:240: _LT_SETUP is expanded from...
m4/libtool.m4:104: LT_INIT is expanded from...
configure.ac:10: the top level
configure.ac:10: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:198: AC_LANG_CONFTEST is expanded from...
../../lib/autoconf/general.m4:2671: _AC_LINK_IFELSE is expanded from...
../../lib/autoconf/general.m4:2681: AC_LINK_IFELSE is expanded from...
m4/libtool.m4:5331: _LT_LINKER_SHLIBS is expanded from...
m4/libtool.m4:5416: _LT_LANG_C_CONFIG is expanded from...
m4/libtool.m4:240: _LT_SETUP is expanded from...
m4/libtool.m4:104: LT_INIT is expanded from...
configure.ac:10: the top level
configure.ac:16: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:198: AC_LANG_CONFTEST is expanded from...
../../lib/autoconf/general.m4:2671: _AC_LINK_IFELSE is expanded from...
../../lib/autoconf/general.m4:2681: AC_LINK_IFELSE is expanded from...
m4/libtool.m4:1100: _LT_SYS_MODULE_PATH_AIX is expanded from...
m4/libtool.m4:6443: _LT_LANG_CXX_CONFIG is expanded from...
m4/libtool.m4:822: _LT_LANG is expanded from...
m4/libtool.m4:811: LT_LANG is expanded from...
configure.ac:16: the top level
configure.ac:29: error: possibly undefined macro: AC_PROG_JAR
If this token and others are legitimate, please use m4_pattern_allow.
See the Autoconf documentation.
configure.ac:30: error: possibly undefined macro: AC_PROG_JAVAC
configure.ac:31: error: possibly undefined macro: AC_JNI_INCLUDE_DIR
configure.ac:150: error: possibly undefined macro: AC_PYTHON_DEVEL
autoreconf: /usr/local/Cellar/autoconf/2.68/bin/autoconf failed with exit status: 1
In file included from replication_manager.cc:70:
In file included from ../hyperdaemon/runtimeconfig.h:35:
/usr/local/include/e/envconfig.h:54:49: error: default arguments cannot be added to an out-of-line definition of a member of a class template
envconfig<T> :: envconfig(const char* envvar, T def = T())
^ ~~~
The clang developers seem to be of the opinion that this is not a clang bug, but c++ that should be illegal:
http://clang-developers.42468.n3.nabble.com/default-arguments-cannot-be-added-to-an-out-of-line-definition-of-a-member-of-a-class-template-tp3182781p3182879.html
Compile Error in hyperdex-0.2b9.
yperdaemon/physical.cc: In member function 'hyperdaemon::physical::returncode hyperdaemon::physical::send(const po6::net::location&, std::auto_ptr<e::buffer>)':
hyperdaemon/physical.cc:223:9: error: 'or_32_nobarrier' is not a member of 'e::atomic'
hyperdaemon/physical.cc:231:9: error: 'and_32_nobarrier' is not a member of 'e::atomic'
hyperdaemon/physical.cc: In member function 'hyperdaemon::physical::returncode hyperdaemon::physical::recv(po6::net::location_, std::auto_ptr<e::buffer>)':
hyperdaemon/physical.cc:340:13: error: 'or_32_nobarrier' is not a member of 'e::atomic'
hyperdaemon/physical.cc:348:9: error: 'and_32_nobarrier' is not a member of 'e::atomic'
hyperdaemon/physical.cc: In member function 'hyperdaemon::physical::returncode hyperdaemon::physical::get_channel(const hazard_ptr&, po6::net::socket, hyperdaemon::physical::channel__)':
hyperdaemon/physical.cc:524:9: error: 'or_32_nobarrier' is not a member of 'e::atomic'
hyperdaemon/physical.cc: In member function 'bool hyperdaemon::physical::work_read(const hazard_ptr&, hyperdaemon::physical::channel_, po6::net::location_, std::auto_ptr<e::buffer>, hyperdaemon::physical::returncode)':
hyperdaemon/physical.cc:633:9: error: 'or_32_nobarrier' is not a member of 'e::atomic'
hyperdaemon/physical.cc: In member function 'bool hyperdaemon::physical::work_write(const hazard_ptr&, hyperdaemon::physical::channel_, hyperdaemon::physical::returncode_)':
hyperdaemon/physical.cc:741:13: error: 'or_32_nobarrier' is not a member of 'e::atomic'
make[1]: *_* [hyperdaemon/libhyperdaemon_la-physical.lo] Error 1
make[1]: Leaving directory `/home/insion/Downloads/hyperdex/hyperdex-0.2b9'
make: *** [all] Error 2
Add methods that remove and return items from a list with list_lpop and list_rpop.
make all-am
make[1]: Entering directory /chroot/local/portage/dev-db/hyperdex-0.4.0/work/hyperdex-0.4.0' CXX hyperdex_binary_test-binary-test.o CXX hyperdex_daemon-daemon.o x86_64-pc-linux-gnu-g++ -DHAVE_CONFIG_H -I. -O2 -pipe -march=native -c -o hyperdex_binary_test-binary-test.o
test -f 'binary-test.cc' || echo './'binary-test.cc x86_64-pc-linux-gnu-g++ -DHAVE_CONFIG_H -I. -I/chroot/local/portage/dev-db/hyperdex-0.4.0/work/hyperdex-0.4.0/hyperspacehashing -I/chroot/local/portage/dev-db/hyperdex-0.4.0/work/hyperdex-0.4.0/hyperdisk -O2 -pipe -march=native -c -o hyperdex_daemon-daemon.o
test -f 'daemon.cc' || echo './'daemon.cc CXX hyperdex/hyperdex_daemon-configuration.o x86_64-pc-linux-gnu-g++ -DHAVE_CONFIG_H -I. -I/chroot/local/portage/dev-db/hyperdex-0.4.0/work/hyperdex-0.4.0/hyperspacehashing -I/chroot/local/portage/dev-db/hyperdex-0.4.0/work/hyperdex-0.4.0/hyperdisk -O2 -pipe -march=native -c -o hyperdex/hyperdex_daemon-configuration.o
test -f 'hyperdex/configuration.cc' || echo './'hyperdex/configuration.cc CXX hyperdex/hyperdex_daemon-configuration_parser.o x86_64-pc-linux-gnu-g++ -DHAVE_CONFIG_H -I. -I/chroot/local/portage/dev-db/hyperdex-0.4.0/work/hyperdex-0.4.0/hyperspacehashing -I/chroot/local/portage/dev-db/hyperdex-0.4.0/work/hyperdex-0.4.0/hyperdisk -O2 -pipe -march=native -c -o hyperdex/hyperdex_daemon-configuration_parser.o
test -f 'hyperdex/configuration_parser.cc' || echo './'hyperdex/configuration_parser.cc CXX hyperdex/hyperdex_daemon-coordinatorlink.o x86_64-pc-linux-gnu-g++ -DHAVE_CONFIG_H -I. -I/chroot/local/portage/dev-db/hyperdex-0.4.0/work/hyperdex-0.4.0/hyperspacehashing -I/chroot/local/portage/dev-db/hyperdex-0.4.0/work/hyperdex-0.4.0/hyperdisk -O2 -pipe -march=native -c -o hyperdex/hyperdex_daemon-coordinatorlink.o
test -f 'hyperdex/coordinatorlink.cc' || echo './'hyperdex/coordinatorlink.cc In file included from /usr/include/e/bufferio.h:32:0, from hyperdex/coordinatorlink.cc:35: /usr/include/po6/io/fd.h: In member function 'void po6::io::fd::close()': /usr/include/po6/io/fd.h:109:9: error: '::close' has not been declared /usr/include/po6/io/fd.h: In member function 'ssize_t po6::io::fd::read(void*, size_t)': /usr/include/po6/io/fd.h:118:12: error: '::read' has not been declared /usr/include/po6/io/fd.h: In member function 'ssize_t po6::io::fd::write(const void*, size_t)': /usr/include/po6/io/fd.h:155:12: error: '::write' has not been declared In file included from hyperclient/hyperclient.h:53:0, from binary-test.cc:42: /usr/include/po6/io/fd.h: In member function 'void po6::io::fd::close()': /usr/include/po6/io/fd.h:109:9: error: '::close' has not been declared /usr/include/po6/io/fd.h: In member function 'ssize_t po6::io::fd::read(void*, size_t)': /usr/include/po6/io/fd.h:118:12: error: '::read' has not been declared /usr/include/po6/io/fd.h: In member function 'ssize_t po6::io::fd::write(const void*, size_t)': /usr/include/po6/io/fd.h:155:12: error: '::write' has not been declared make[1]: *** [hyperdex_binary_test-binary-test.o] Error 1 make[1]: *** Waiting for unfinished jobs.... make[1]: *** [hyperdex/hyperdex_daemon-coordinatorlink.o] Error 1 make[1]: Leaving directory
/chroot/local/portage/dev-db/hyperdex-0.4.0/work/hyperdex-0.4.0'
make: *** [all] Error 2
gcc version 4.7.1 (Gentoo 4.7.1 p1.2, pie-0.5.3)
Every op that exists for int64 should exist for float as well.
Immediate consistency makes operations easy to reason about, however without the ability to conditionally modify, some of it's advantage is lost. Would it be possible to extend the public API to return, and accept for comparison, some hash value of a current dataset entry such that conditional update can be performed?
In each case the hash would be a small, base64 encoded, constant length hash of the entries attributes. The following should be valid for all modifiers, i.e. insertion and deletion.
An example interaction may be:
Insert entry, now returns special _hash
value.
>>> import hyperclient
>>> c = hyperclient.Client('127.0.0.1', 1234)
>>> c.insert('phonebook', 'jsmith1', {'first': 'John', 'last': 'Smith', 'phone': 6075551024})
True
>>> c.lookup('phonebook', 'jsmith1')
phonebook(first='John', last='Smith', phone='0\x8d!j\x01', username='jsmith1', _hash: 'a6n9bf')
Attempt to modify with incorrect hash fails:
>>> c.insert('phonebook', 'jsmith1', {'phone': 6075552048}, 'zzz9bf')
False
>>> c.lookup('phonebook', 'jsmith1')
phonebook(first='John', last='Smith', phone='0\x8d!j\x01', _hash: 'a6n9bf')
Attempt to modify with correct hash succeeds:
>>> c.insert('phonebook', 'jsmith1', {'phone': 6075552048}, 'a6n9bf')
True
>>> c.lookup('phonebook', 'jsmith1')
phonebook(first='John', last='Smith', phone='0\x91!j\x01', _hash: 'q8bbo0')
Attempt to modify with null hash succeeds:
>>> c.insert('phonebook', 'jsmith1', {'phone': 6075551024})
True
>>> c.lookup('phonebook', 'jsmith1')
phonebook(first='John', last='Smith', phone='0\x8d!j\x01', _hash: 'q8bbo0')
Attempt to modify with empty hash fails as key already in use:
>>> c.insert('phonebook', 'jsmith1', {'first': 'Johnny', 'last': 'Smithton', 'phone': 923478383}, "")
False
>>> c.lookup('phonebook', 'jsmith1')
phonebook(first='John', last='Smith', phone='0\x8d!j\x01', _hash: 'q8bbo0')
Attempt to modify with null hash succeeds, overwriting the current entry:
>>> c.insert('phonebook', 'jsmith1', {'first': 'Johnny', 'last': 'Smithton', 'phone': 6075551024})
False
>>> c.lookup('phonebook', 'jsmith1')
phonebook(first='Johnny', last='Smithton', phone='0\x6f!j\x01', _hash: '93jj7d')
I think that this adds a huge use-case for the store.
When vals
is a list of numbers, the following is possible:
>>> c.list_rpush('keyvalues','1234',{'vals': 54.2355})
>>> False
What is expected is:
>>> c.list_rpush('keyvalues','1234',{'vals': 54.2355})
Exception: TypeError
54.2355 is not an integer, so a TypeError should be thrown.
There are several things wrong when trying to install the binary package on plain Ubuntu 12.04 following the instructions at http://hyperdex.org/download :
A) Ubuntu does not support a root user by default so the correct commands for installing the package would be something like:
wget -O - http://ubuntu.hyperdex.org/hyperdex.gpg.key | sudo apt-key add -
sudo wget -O /etc/apt/sources.list.d/hyperdex.list http://ubuntu.hyperdex.org/hyperdex.list
sudo aptitude update
B) aptitude is not installed by default, so you would need:
sudo apt-get install aptitude
C) The package libgoogle-glog0 is not supported anymore and would require adding untrusted PPAs to install it.
CXX hyperdex_binary_test-binary-test.o
binary-test.cc: In function ‘int main(int, char*)’:
binary-test.cc:141:35: error: call of overloaded ‘pack64le(uint32_t&, uint8_t [8])’ is ambiguous
binary-test.cc:141:35: note: candidates are:
/opt/hyperdex/include/e/endian.h:103:1: note: uint8_t e::pack64le(uint64_t, uint8_t_)
/opt/hyperdex/include/e/endian.h:286:1: note: uint8_t_ e::pack64le(int64_t, uint8_t_)
make[1]: *_* [hyperdex_binary_test-binary-test.o] Error 1
make[1]: Leaving directory `/home/nick/proj/HyperDex'
make: *** [all] Error 2
$ autoreconf -i -v;
autoreconf2.50: Entering directory .' autoreconf2.50: configure.ac: not using Gettext autoreconf2.50: running: aclocal -I m4 ${ACLOCAL_FLAGS} autoreconf2.50: configure.ac: tracing autoreconf2.50: running: libtoolize --copy libtoolize: putting auxiliary files in
.'.
libtoolize: copying file ./ltmain.sh' libtoolize: putting macros in AC_CONFIG_MACRO_DIR,
m4'.
libtoolize: copying file m4/libtool.m4' libtoolize: copying file
m4/ltoptions.m4'
libtoolize: copying file m4/ltsugar.m4' libtoolize: copying file
m4/ltversion.m4'
libtoolize: copying file `m4/lt~obsolete.m4'
autoreconf2.50: running: /usr/bin/autoconf
configure.ac:31: error: possibly undefined macro: AC_PROG_JAR
If this token and others are legitimate, please use m4_pattern_allow.
See the Autoconf documentation.
configure.ac:32: error: possibly undefined macro: AC_PROG_JAVAC
configure.ac:33: error: possibly undefined macro: AC_JNI_INCLUDE_DIR
configure.ac:151: error: possibly undefined macro: AC_PYTHON_DEVEL
autoreconf2.50: /usr/bin/autoconf failed with exit status: 1
I success configuare the hyperdex-0.2b8 without error . But When I 'make',the error happen.
"
make all-am
make[1]: Entering directory /usr/home/yanglong1/hyperdex/hyperdex-0.2b8' CXX hyperspacehashing/libhyperspacehashing_la-cfloat.lo hyperspacehashing/cfloat.cc: In function ‘uint64_t hyperspacehashing::cfloat(uint64_t, unsigned int)’: hyperspacehashing/cfloat.cc:61: warning: converting to ‘uint64_t’ from ‘double’ CXX hyperspacehashing/libhyperspacehashing_la-hashes.lo hyperspacehashing/hashes.cc: In function ‘uint64_t hyperspacehashing::lendian(const e::slice&)’: hyperspacehashing/hashes.cc:56: error: ‘le64toh’ was not declared in this scope make[1]: *** [hyperspacehashing/libhyperspacehashing_la-hashes.lo] Error 1 make[1]: Leaving directory
/usr/home/yanglong1/hyperdex/hyperdex-0.2b8'
make: *** [all] Error 2
"
Need Help.
I use Centos 5.4 x64.
Thank You.
autoreconf will generate a non-functional configure script because of 'm4/anal_warnings_cxx.m4' -- running the resulting configure script will fail with a syntax error. I had to delete 'm4/anal_warnings_cxx.m4' and run autoreconf again to get a working configure script.
I'd like to see setrange
and getrange
added to the API (functionally equivalent to the respective redis operations).
These operations are needed for a k/v-store to support segmented mmap()-style usage.
I tried to start with ./hyperdex-coordinator -?
, but I get the aforementioned error message. I also tried the command in the tutorial: hyperdex-coordinator --control-port 6970 --host-port 1234 --logging debug
As far as I understand, this should work? I installed all the prerequisites (they got installed into /usr/local/lib64), although I'm a little confused about the fact that the link to libpopt here http://hyperdex.org/download/ leads to rpm5. I have libpopt0 installed though, I'm on opensuse 12.1.
I installed hyperdex from the tarball into a folder in my home directory.
also more, I tried to start the daemon anyway: Where does hyperdex look for the libraries? ./hyperdex-daemon: error while loading shared libraries: libcityhash.so.0: cannot open shared object file: No such file or directory
It's installed into /usr/local/lib64
Per tutorial, got the following errors to run "autoreconf" to prep for build.
autoreconf -i -v
autoreconf: Entering directory .' autoreconf: configure.ac: not using Gettext autoreconf: running: aclocal -I m4 ${ACLOCAL_FLAGS} autoreconf: configure.ac: tracing autoreconf: configure.ac: not using Libtool autoreconf: running: /usr/bin/autoconf autoreconf: running: /usr/bin/autoheader autoreconf: running: automake --add-missing --copy --no-force Makefile.am:38: Libtool library used but
LIBTOOL' is undefined
Makefile.am:38: The usual way to define LIBTOOL' is to add
AC_PROG_LIBTOOL'
Makefile.am:38: to configure.ac' and run
aclocal' and autoconf' again. Makefile.am:38: If
AC_PROG_LIBTOOL' is in `configure.ac', make sure
Makefile.am:38: its definition is in aclocal's search path.
autoreconf: automake failed with exit status: 1
Please advise what can be missing.
By the way, build tools autoconf, automake, and libtool have the following versions on debian.
autoconf, 2.67-2, automatic configure script builder
automake, 1:1.11.1-1, A tool for generating GNU Standards-compliant Makefiles
libltdl7, 2.2.6b-2, A system independent dlopen wrapper for GNU libtool
It looks like that on my debian, I have libltdl7 instead of "libtool". Does it have to be exact?
Show me the river to follow? http://is.gd/Py6OcG http://youtu.be/iZFI7ISEuWw
Reading package lists... Done
Building dependency tree
Reading state information... Done
Reading extended state information
Initializing package states... Done
The following packages are BROKEN:
hyperdex-daemon python-hyperclient python-hypercoordinator
The following NEW packages will be installed:
hyperdex libcityhash0{a} libe0{a} libhyperclient0{a} libhyperdisk0{a} libhyperspacehashing0{a} python-pyparsing{a}
0 packages upgraded, 10 newly installed, 0 to remove and 201 not upgraded.
Need to get 1,342kB of archives. After unpacking 6,132kB will be used.
The following packages have unmet dependencies:
hyperdex-daemon: Depends: libgoogle-glog0 which is a virtual package.
Depends: libpopt0 (>= 1.16) but 1.15-1 is installed.
Depends: libstdc++6 (>= 4.6) but 4.4.3-4ubuntu5 is installed and it is kept back.
python-hyperclient: Depends: python (>= 2.7) but 2.6.5-0ubuntu1 is installed.
python-hypercoordinator: Depends: python (>= 2.7) but 2.6.5-0ubuntu1 is installed.
The following actions will resolve these dependencies:
Keep the following packages at their current version:
hyperdex [Not Installed]
hyperdex-daemon [Not Installed]
python-hyperclient [Not Installed]
python-hypercoordinator [Not Installed]
Score is -9846
Accept this solution? [Y/n/q/?] q
Abandoning all efforts to resolve these dependencies.
Abort.
root@srv1 /home/kunthar # uname -a
Linux srv1.sintez.com.tr 2.6.32-23-server #37-Ubuntu SMP Fri Jun 11 09:11:11 UTC 2010 x86_64 GNU/Linux
root@srv1 /home/kunthar # cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=10.04
DISTRIB_CODENAME=lucid
DISTRIB_DESCRIPTION="Ubuntu 10.04.1 LTS"
While trying to compile Libe I get the following error:
Jacobs-MacBook-Air-2:libe-0.2.5 jacob$ make
make all-recursive
Making all in benchmarks
CXX lockfree_hash_map.o
lockfree_hash_map.cc: In function ‘void worker_thread(e::lockfree_hash_map<long long unsigned int, long long unsigned int, id>*)’:
lockfree_hash_map.cc:143: error: ‘drand48_data’ was not declared in this scope
lockfree_hash_map.cc:143: error: expected `;' before ‘buf’
lockfree_hash_map.cc:147: error: invalid conversion from ‘_opaque_pthread_t*’ to ‘short unsigned int’
lockfree_hash_map.cc:148: error: ‘buf’ was not declared in this scope
lockfree_hash_map.cc:148: error: ‘seed48_r’ was not declared in this scope
lockfree_hash_map.cc:154: error: ‘lrand48_r’ was not declared in this scope
lockfree_hash_map.cc:163: error: ISO C++ forbids comparison between pointer and integer
lockfree_hash_map.cc:175: error: invalid conversion from ‘_opaque_pthread_t*’ to ‘long long unsigned int’
lockfree_hash_map.cc:175: error: initializing argument 2 of ‘bool e::lockfree_hash_map<K, V, H>::insert(const K&, const V&) [with K = long long unsigned int, V = long long unsigned int, uint64_t (* H)(const K&) = id]’
make[2]: *** [lockfree_hash_map.o] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2
Jacobs-MacBook-Air-2:libe-0.2.5 jacob$
I have no idea to make of it, but any help would be appreciated (if it's a problem with my system) or hopefully the bug can be found (if it's a problem with the codebase)
thanks!
The latest e ( commit 614ad1452d ) doesn't seem to have the file ... include/e/endian.h that the latest Hyperdex ( commit 29b8a02) tries to include.
CXX hyperclient/libhyperclient_la-hyperclient.lo
hyperclient/hyperclient.cc:37:22: fatal error: e/endian.h: No such file or directory
compilation terminated.
make[1]: *** [hyperclient/libhyperclient_la-hyperclient.lo] Error 1
make[1]: Leaving directory `/home/nick/proj/HyperDex'
make: *** [all] Error 2
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.