btrdb / btrdb-python Goto Github PK
View Code? Open in Web Editor NEWBTrDB bindings for Python
Home Page: https://btrdb.readthedocs.io/en/latest/
License: Other
BTrDB bindings for Python
Home Page: https://btrdb.readthedocs.io/en/latest/
License: Other
The colls parameter is apparently used for pagination.
def listCollections(self, prefix):
colls = (prefix,)
maximum = 10
got = maximum
while got == maximum:
startingAt = colls[-1]
colls = self.ep.listCollections(prefix, colls, maximum)
for coll in colls:
yield coll
got = len(colls)
However, Python always throws a Type error
TypeError: ('blahblahblah',) has type <class 'tuple'>, but expected one of: (<class 'bytes'>, <class 'str'>) for field ListCollectionsParams.startWith
This issue should be debated.
The method is refreshMeta() whereas the GoLang version is RefreshMeta(). In Python, this should be refresh_meta(). It would be "more pythonic" to follow the formatting standards. Should the python library conform to python formatting standards or try to match the Go Lang version?
There are some other issues with spacing that are probably just being a bit nitpicky.
In general, handling nanoseconds in Python is not terribly pleasant. It would make sense to create some tooling or helper classes around this issue. As time stamps in nanoseconds are incredibly large, it is easy to screw up the order of magnitude?
Do we use datetime? Time? Arrow? Pandas?
Some possible helper functions are below.
def ConvertDateTimeToEpoch_ns(dt):
"""
converts the datetime object to epoch time in nanoseconds
"""
return (int(dt.strftime('%s'))*1e6 + dt.microsecond)*1e3
def ConvertEpoch_nsToDateTime(ns):
"""
converts the Epoch time in nanoseconds into a datetime object
"""
return (datetime.fromtimestamp(ns/1e9).strftime('%c'))
It would be nice to be able to decrement by an "hour" or "minute" or "day"
This method is not fully implemented
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
<ipython-input-22-c0d61ae0f3fe> in <module>()
----> 1 for x in s.changes(v-1, v, 1):
2 print(x)
~/anaconda/lib/python3.6/site-packages/btrdb4-4.4.5-py3.6.egg/btrdb4/__init__.py in changes(self, fromVersion, toVersion, resolution)
428 for crlist, version in crs:
429 for cr in crlist:
--> 430 yield ChangeRange.fromProto(cr), version
431
432 def flush(self):
NameError: name 'ChangeRange' is not defined
Variable types is a bit unforgiving in the library.
For example - nanoseconds must be in integers. We could simply cast nanosecond parameters as integers within the api methods instead of throwing errors. There could be a performance penalty for doing this.
There are also some instances where the user has to take a string and turn it into a sequence of bytes in a particular character encoding. This can easily be done inside the API using either of the following to generate b'bytes':
u'something'.encode('utf-8')
bytes(u'something', 'utf-8')
download_url = 'https://github.com/SoftwareDefinedBuildings/btrdb4-python/tarball/0.0.1',
If you pick a point and then go the wrong way such that there is no data in that direction, BTrDB returns
BTrDBError [401] no such point
This should not be the case. It would be much smarter to return a None and have the API method catch the error and return none.
In cases where HOME
is not set (e.g. in our case, when btrdb
is imported in a webapp controlled by supervisorctl
), importing btrdb
fails at utils/credentials.py:30
CREDENTIALS_PATH = os.path.join(os.environ["HOME"], CONFIG_DIR, CREDENTIALS_FILENAME)
This can be reproduced as follows:
# bash
unset HOME
then
# in python
import btrdb
Which produces:
In [1]: import btrdb
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
<ipython-input-1-36165869e3ee> in <module>
----> 1 import btrdb
~/Preteckt/tools/preteckt_dash/env/lib/python3.6/site-packages/btrdb/__init__.py in <module>
20 from btrdb.exceptions import ConnectionError
21 from btrdb.version import get_version
---> 22 from btrdb.utils.credentials import credentials_by_profile, credentials
23 from btrdb.stream import MINIMUM_TIME, MAXIMUM_TIME
24
~/Preteckt/tools/preteckt_dash/env/lib/python3.6/site-packages/btrdb/utils/credentials.py in <module>
28 CONFIG_DIR = ".predictivegrid"
29 CREDENTIALS_FILENAME = "credentials.yaml"
---> 30 CREDENTIALS_PATH = os.path.join(os.environ["HOME"], CONFIG_DIR, CREDENTIALS_FILENAME)
31
32 ##########################################################################
/usr/lib/python3.6/os.py in __getitem__(self, key)
667 except KeyError:
668 # raise KeyError with the original key value
--> 669 raise KeyError(key) from None
670 return self.decodevalue(value)
671
KeyError: 'HOME'
There is of course a workaround to set the HOME
var by hand, but it seems like this shouldn't be a requirement.
Search functionality for streams is currently less than ideal.
It would be good to have wildcard or more general purpose string matching available. This could be built into the Python library or could be built into BTrDB. It is probably better to put it into BTrDB.
It may make sense to have some basic utility oriented functionality built into the API or as an additional library.
The 4 in the current name is misleading. We should drop it from the name of the project repo.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.