quandl / quandl-python Goto Github PK
View Code? Open in Web Editor NEWLicense: MIT License
License: MIT License
Hi.
I tried to import quandl but I received this Traceback error: "ImportError: No module named 'inflection'" I reviewed your folder and found no file called inflection.py.
This is the message detail:
Traceback (most recent call last):
File "<pyshell#41>", line 1, in
import quandl
File "C:\Python35\lib\site-packages\quandl_init_.py", line 7, in
from .model.database import Database
File "C:\Python35\lib\site-packages\quandl\model\database.py", line 11, in
from quandl.connection import Connection
File "C:\Python35\lib\site-packages\quandl\connection.py", line 5, in
from .util import Util
File "C:\Python\Python35\lib\site-packages\quandl\util.py", line 1, in
from inflection import parameterize
ImportError: No module named 'inflection
Hi
I came across this issue while trying to install quandl (python):
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c build/temp.linux-x86_64-2.7/_openssl.c -o build/temp.linux-x86_64-2.7/build/temp.linux-x86_64-2.7/_openssl.o
build/temp.linux-x86_64-2.7/_openssl.c:12:24: fatal error: pyconfig.h: No such file or directory
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
Command "/usr/bin/python -u -c "import setuptools, tokenize;file='/tmp/pip-build-1wgahm/cryptography/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-RV3Adm-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-1wgahm/cryptography/
Is the dependency on ipython ("from IPython import embed") needed ? it does not look used anywhere in the code.
I tried to install in a virtual env a but quandl is getting installed in global env
Please rectify this problem
Sorry you can kindly close this it was a set up issue in my venv
import quandl
quandl.ApiConfig.api_key = "mysecretkey"
Save API Key string in a local text file, e.g. $HOME/.quandl_apikey
quandl.apikey_set("mysecretkey")
or a specified file, e.g.
quandl.apikey_set("mysecretkey", filepath="/data/.quandl_apikey")
Use cases
/data/
to external storage (It's really painful to inject it with run command, and storing keys and tokens inside container is bad practice)https://github.com/ulf1/updoon/blob/master/updoon/quandl_apikey.py
this would read the file with the api like
import quandl
import updoon
quandl.ApiConfig.api_key = updoon.quandl_apikey()
Add save_key
and read_key
to api_config.py
, e.g.
class ApiConfig:
....
def read_key(filepath=None):
...
ApiConfig.api_key = ...
to do
import quandl
quandl.read_key()
...
instead of exposing the API key within a script.
Hello,
conda install fetch 2.8.9 version
$ conda --version
conda 4.1.11
$ python --version
Python 3.5.2 :: Anaconda custom (x86_64)
$ conda install quandl
Fetching package metadata .......
Solving package specifications: ..........
Package plan for installation in environment /Users/femto/anaconda:
The following packages will be downloaded:
package | build
---------------------------|-----------------
quandl-2.8.9 | py35_0 11 KB
The following NEW packages will be INSTALLED:
quandl: 2.8.9-py35_0
Proceed ([y]/n)? y
Fetching packages ...
quandl-2.8.9-p 100% |############################################################################################################################################################| Time: 0:00:00 787.71 kB/s
Extracting packages ...
[ COMPLETE ]|###############################################################################################################################################################################| 100%
Linking packages ...
[ COMPLETE ]|###############################################################################################################################################################################| 100%
$ ipython
Python 3.5.2 |Anaconda custom (x86_64)| (default, Jul 2 2016, 17:52:12)
Type "copyright", "credits" or "license" for more information.
IPython 5.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
In [1]: import quandl
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
<ipython-input-1-ab7fa5ad38a5> in <module>()
----> 1 import quandl
ImportError: No module named 'quandl'
In [2]: import Quandl
In [3]: Quandl.__version__
Out[3]: '2.8.9'
In [4]: import pandas as pd
In [5]: pd.set_option("max_rows", 10)
In [6]: Quandl.get("ISM/MAN_PMI")
Out[6]:
Index
Date
1948-01-01 51.7
1948-02-01 50.2
1948-03-01 43.3
1948-04-01 45.4
1948-05-01 49.5
... ...
2016-03-01 51.8
2016-04-01 50.8
2016-05-01 51.3
2016-06-01 53.2
2016-07-01 52.6
[823 rows x 1 columns]
I thought that current version was 3.
Kind regards
Hi All,
I get the following error on installing Quandl 3.0.1 for Python 2.7.
Processing dependencies for Quandl==3.0.1
Searching for pyasn1
Reading https://pypi.python.org/simple/pyasn1/
Download error on https://pypi.python.org/simple/pyasn1/: [Errno 11004] getaddri
nfo failed -- Some packages may not be found!
Couldn't find index page for 'pyasn1' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading https://pypi.python.org/simple/
Download error on https://pypi.python.org/simple/: [Errno 11004] getaddrinfo fai
led -- Some packages may not be found!
No local packages or download links found for pyasn1
error: Could not find suitable distribution for Requirement.parse('pyasn1')
Please suggest if there is an alternative.
Thanks,
Best,
Pruthvi
I'm not sure if this issue has been raised before elsewhere.
I was curious to learn why some of the free Quandl economic databases often update without the most recent data. For example, the European Commission Business and Consumer Surveys Quandl dataset doesn't currently include the most recent data (February, 2018). It looks like Quandl is updating on the correct release date, but the new data isn't available until subsequent updates.
I was also having similar issues with the Institute for Supply Management in January, however, this no longer seems to be an issue.
Thanks in advance.
#117 introduced a bug in regards to Python 2 compatibility since pathlib
is Python 3.4+ only.
Python 2.7.15rc1 (default, Nov 12 2018, 14:31:15)
[GCC 7.3.0] on linux2
>>> import quandl
>>> quandl.read_key()
Traceback (most recent call last):
...
ImportError: No module named pathlib
>>> quandl.save_key('secret...')
Traceback (most recent call last):
...
ImportError: No module named pathlib
Currently, the only way to hit these import statements is to not pass a filename
argument to either function (or pass filename=None
). Here are the two sections causing the issue:
quandl-python/quandl/api_config.py
Lines 15 to 18 in 3314981
quandl-python/quandl/api_config.py
Lines 26 to 29 in 3314981
This problem would likely have been caught earlier if the imports were at the top of the file as mentioned in PEP8.
quandl-python/quandl/__init__.py
Line 3 in 3314981
A simple fix seems to be indicated in the Python docs themselves:
Return a new path object representing the userโs home directory (as returned by os.path.expanduser() with ~ construct)
Python 3.7.1 (default, Oct 25 2018, 21:25:21)
[GCC 7.3.0] on linux
>>> import os
>>> import pathlib
>>> os.path.expanduser('~') == str(pathlib.Path.home())
True
There is a problem with simply replacing str(pathlib.Path.home())
with os.path.expanduser('~')
in these two lines though. The string concatenation is prepending forward slash in front of .quandl_apikey
which (if I remember correctly) is not compatible with Windows which uses backslashes.
filename = str(pathlib.Path.home()) + "/.quandl_apikey"
^
I'm a fan of pathlib
and one way to solve this would be like this:
>>> pathlib.Path.home() / ".quandl_apikey"
PosixPath('/home/.../.quandl_apikey')
But since Python 2 compatibility is the issue, using os.path.join
seems like an appropriate fix.
>>> p = pathlib.Path.home() / ".quandl_apikey"
>>> str(p) == os.path.join(os.path.expanduser('~'), '.quandl_apikey')
True
There is also a smaller bug where the try/except
in read_key
is looking for the wrong exception. In Python 2, open
raises an IOError
if a file isn't found. In Python 3, an OSError
is raised.
Neither of these exceptions have a parent exception of ValueError
though.
Python 2.7.15rc1 (default, Nov 12 2018, 14:31:15)
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> try:
... with open('doesnt_exist.txt', 'r') as f:
... pass
... except ValueError:
... raise Exception("File '{:s}' not found.".format(filename))
...
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
IOError: [Errno 2] No such file or directory: 'doesnt_exist.txt'
Python 3.7.1 (default, Oct 25 2018, 21:25:21)
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> try:
... with open('doesnt_exist.txt', 'r') as f:
... pass
... except ValueError:
... raise Exception("File '{:s}' not found.".format(filename))
...
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
FileNotFoundError: [Errno 2] No such file or directory: 'doesnt_exist.txt'
In my opinion, these exceptions are already very clear about what went wrong and it seems unnecessary to catch them in read_key
just to simply re-raise them with a custom message.
Ignoring the fact that these two functions are mutating the state of a class and seem like they could be class methods instead... maybe something like this?
import os
class ApiConfig:
api_key = None
api_protocol = 'https://'
api_base = '{}www.quandl.com/api/v3'.format(api_protocol)
api_version = None
page_limit = 100
use_retries = True
number_of_retries = 5
retry_backoff_factor = 0.5
max_wait_between_retries = 8
retry_status_codes = [429] + list(range(500, 512))
def save_key(apikey, filename=None):
if filename is None:
filename = os.path.join(os.path.expanduser('~'), '.quandl_apikey')
with open(filename, 'w') as f:
f.write(apikey)
ApiConfig.api_key = apikey
def read_key(filename=None):
if filename is None:
filename = os.path.join(os.path.expanduser('~'), '.quandl_apikey')
with open(filename, 'r') as f:
apikey = f.read()
if not apikey:
raise ValueError("File '{:s}' is empty.".format(filename))
ApiConfig.api_key = apikey
Hi,
Does the Quandl API support Intraday Data? 5minutes, hourly data?
For example :
from alpha_vantage.timeseries import TimeSeries
data, meta_data = ts.get_intraday(symbol=AAPL, interval='60min', outputsize='compact')
API document only has the details on how to retrieve EOD data with the date range.
https://docs.quandl.com/docs/python-time-series#section-make-a-filtered-time-series-call
data = quandl.get("FRED/GDP")
print(data.head())
With correct api key (already renewed api key and checked with curl the same request as https://www.quandl.com/api/v3/datasets/FRED/GDP/data.json?api_key=XXXXXXXXXX and it returns correct formatted json) but Quandl 3.4.3 does return:
JSONDecodeError Traceback (most recent call last)
/usr/local/lib/python3.5/dist-packages/quandl/connection.py in parse(cls, response)
54 try:
---> 55 return response.json()
56 except ValueError:
/usr/local/lib/python3.5/dist-packages/requests/models.py in json(self, **kwargs)
895 pass
--> 896 return complexjson.loads(self.text, **kwargs)
897
/usr/lib/python3.5/json/init.py in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
318 parse_constant is None and object_pairs_hook is None and not kw):
--> 319 return _default_decoder.decode(s)
320 if cls is None:
/usr/lib/python3.5/json/decoder.py in decode(self, s, _w)
338 """
--> 339 obj, end = self.raw_decode(s, idx=_w(s, 0).end())
340 end = _w(s, end).end()
/usr/lib/python3.5/json/decoder.py in raw_decode(self, s, idx)
356 except StopIteration as err:
--> 357 raise JSONDecodeError("Expecting value", s, err.value) from None
358 return obj, end
JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
QuandlError Traceback (most recent call last)
in
----> 1 data = quandl.get("FRED/GDP")
2 print(data.head())
/usr/local/lib/python3.5/dist-packages/quandl/get.py in get(dataset, **kwargs)
46 if dataset_args['column_index'] is not None:
47 kwargs.update({'column_index': dataset_args['column_index']})
---> 48 data = Dataset(dataset_args['code']).data(params=kwargs, handle_column_not_found=True)
49 # Array
50 elif isinstance(dataset, list):
/usr/local/lib/python3.5/dist-packages/quandl/model/dataset.py in data(self, **options)
45 updated_options = Util.merge_options('params', params, **options)
46 try:
---> 47 return Data.all(**updated_options)
48 except NotFoundError:
49 if handle_not_found_error:
/usr/local/lib/python3.5/dist-packages/quandl/operations/list.py in all(cls, **options)
12 options['params'] = {}
13 path = Util.constructed_path(cls.list_path(), options['params'])
---> 14 r = Connection.request('get', path, **options)
15 response_data = r.json()
16 Util.convert_to_dates(response_data)
/usr/local/lib/python3.5/dist-packages/quandl/connection.py in request(cls, http_verb, url, **options)
34 abs_url = '%s/%s' % (ApiConfig.api_base, url)
35
---> 36 return cls.execute_request(http_verb, abs_url, **options)
37
38 @classmethod
/usr/local/lib/python3.5/dist-packages/quandl/connection.py in execute_request(cls, http_verb, url, **options)
42 response = func(url, **options)
43 if response.status_code < 200 or response.status_code >= 300:
---> 44 cls.handle_api_error(response)
45 else:
46 return response
/usr/local/lib/python3.5/dist-packages/quandl/connection.py in handle_api_error(cls, resp)
59 @classmethod
60 def handle_api_error(cls, resp):
---> 61 error_body = cls.parse(resp)
62
63 # if our app does not form a proper quandl_error response
/usr/local/lib/python3.5/dist-packages/quandl/connection.py in parse(cls, response)
55 return response.json()
56 except ValueError:
---> 57 raise QuandlError(http_status=response.status_code, http_body=response.text)
58
59 @classmethod
QuandlError: (Status 403) Something went wrong. Please try again. If you continue to have problems, please contact us at [email protected].
I'm getting the following warning on issuing the following conda command:
conda update --all
Fetching package metadata .............
Solving package specifications: .
Warning: 2 possible package resolutions (only showing differing packages):
- defaults::quandl-3.2.0-py36h30f7a5e_0, defaults::requests-2.14.2-py36_0
- defaults::quandl-3.1.0-py36_0, defaults::requests-2.18.4-py36h4371aae_1
Here's my conda info:
Current conda install:
platform : win-64
conda version : 4.3.30
conda is private : False
conda-env version : 4.3.30
conda-build version : 3.0.30
python version : 3.6.3.final.0
requests version : 2.14.2
root environment : D:\Users\803375527\AppData\Local\Continuum\anaconda3 (writable)
default environment : D:\Users\803375527\AppData\Local\Continuum\anaconda3
envs directories : D:\Users\803375527\AppData\Local\Continuum\anaconda3\envs
C:\Users\803375527\AppData\Local\conda\conda\envs
C:\Users\803375527\.conda\envs
package cache : D:\Users\803375527\AppData\Local\Continuum\anaconda3\pkgs
C:\Users\803375527\AppData\Local\conda\conda\pkgs
channel URLs : https://repo.continuum.io/pkgs/main/win-64
https://repo.continuum.io/pkgs/main/noarch
https://repo.continuum.io/pkgs/free/win-64
https://repo.continuum.io/pkgs/free/noarch
https://repo.continuum.io/pkgs/r/win-64
https://repo.continuum.io/pkgs/r/noarch
https://repo.continuum.io/pkgs/pro/win-64
https://repo.continuum.io/pkgs/pro/noarch
https://repo.continuum.io/pkgs/msys2/win-64
https://repo.continuum.io/pkgs/msys2/noarch
config file : C:\Users\803375527\.condarc
netrc file : None
offline mode : False
user-agent : conda/4.3.30 requests/2.14.2 CPython/3.6.3 Windows/10 Windows/10.0.14393
administrator : False
In fact, adjusted data is identical to non-adjusted data. Consider this example (stock = CGNX):
Here is output from 2017-11-20 to 2017-12-18. https://imgur.com/a/PAd4s
This is data pulled like so:
test = quandl.get('WIKI/CGNX')
Split ratio remains at 1.0 throughout, and adjusted & unadjusted values are identical. However there is ample evidence of a stock split around the end of October, and any properly adjusted data shows this:
https://imgur.com/a/qGUfw (from Yahoo Finance, all values below $100).
Quandl doesn't support Python 2.6 anyway, so there is no reason why to bother with adding an additional dependency on unittest2
.
This patch fixes the situation.
I am seeing
incorrect Quandl code error
when the argument of quandl.get() function has special characters.
code snippet:
import quandl
mydata = quandl.get("NSE/BAJAJ-AUTO")
Issue:
quandl.errors.quandl_error.NotFoundError: (Status 404) (Quandl Error QECx02) You have submitted an incorrect Quandl code. Please check your Quandl codes and try again.
Hello,
I'm reading you code...
if I where you I won't write in https://github.com/quandl/quandl-python/blob/feature-AP-243/v3-client/quandl/connection.py
if code_letter == 'L':
klass = LimitExceededError
elif code_letter == 'M':
klass = InternalServerError
elif code_letter == 'A':
klass = AuthenticationError
elif code_letter == 'P':
klass = ForbiddenError
elif code_letter == 'S':
klass = InvalidRequestError
elif code_letter == 'C':
klass = NotFoundError
elif code_letter == 'X':
klass = ServiceUnavailableError
else:
klass = QuandlError
but
d_klass = {
'L': LimitExceededError,
'M': InternalServerError,
'A': AuthenticationError,
'P': ForbiddenError,
'S': InvalidRequestError,
'C': NotFoundError,
'X': ServiceUnavailableError
}
klass_other = QuandlError
try:
klass = d_klass[code_letter]
except KeyError:
klass = QuandlError
Kind regards
A conda recipe for quandl-python could be helpful.
Docs
conda skeleton pypi quandl-python
Examples
If I download data using this Quandl.get() method and then subsequently plot the data the date range is inverted. This does not happen if I initialize a pandas.core.series.TimeSeries myself.
%pylab inline
import Quandl
import pandas as pd
tsx = Quandl.get("YAHOO/INDEX_GSPTSE")
print type(tsx['Open'])
print tsx['Open']
tsx['Open'].plot()
ds = pd.date_range(start='2012-01-01', periods=100)[::-1]
x = pd.Series(range(100), index=ds)
print type(x)
x.head()
x.plot()
Since some days, the builds are failing at openSUSES build system with this error:
[ 32s] ======================================================================
[ 32s] FAIL: test_get_merged_dataset_data_to_list (test.test_merged_dataset.GetMergedDatasetTest)
[ 32s] ----------------------------------------------------------------------
[ 32s] Traceback (most recent call last):
[ 32s] File "/home/abuild/rpmbuild/BUILD/quandl-python-3.3.0/test/test_merged_dataset.py", line 206, in test_get_merged_dataset_data_to_list
[ 32s] self.assertItemsEqual(results[index], expected_item)
[ 32s] AssertionError: Expected, but missing:
[ 32s] [1436572800000000000L]
[ 32s] Unexpected, but present:
[ 32s] [datetime.datetime(2015, 7, 11, 0, 0)]
[ 32s] -------------------- >> begin captured logging << --------------------
[ 32s] urllib3.connectionpool: DEBUG: Starting new HTTPS connection (1): www.quandl.com
[ 32s] urllib3.connectionpool: DEBUG: https://www.quandl.com:443 "GET /api/v3/datasets/NSE/OIL/data?order=asc HTTP/1.1" 200 276
[ 32s] urllib3.connectionpool: DEBUG: Starting new HTTPS connection (1): www.quandl.com
[ 32s] urllib3.connectionpool: DEBUG: https://www.quandl.com:443 "GET /api/v3/datasets/SINGLE/COLUMN/data?column_index=1&order=asc HTTP/1.1" 200 227
[ 32s] urllib3.connectionpool: DEBUG: Starting new HTTPS connection (1): www.quandl.com
[ 32s] urllib3.connectionpool: DEBUG: https://www.quandl.com:443 "GET /api/v3/datasets/WIKI/MSFT/data?order=asc HTTP/1.1" 200 276
[ 32s] --------------------- >> end captured logging << ---------------------
[ 32s]
[ 32s] ----------------------------------------------------------------------
Full log is here: https://build.opensuse.org/build/devel:languages:python/openSUSE_Tumbleweed/x86_64/python-Quandl/_log
I suspect an incompatibility with some dependency (probably dateutil?), these are the ones used:
python3-base-3.6.5-3.1
python3-six-1.11.0-66.7
python3-idna-2.6-10.5
python3-3.6.5-3.1
python3-python-dateutil-2.6.1-14.14
python3-appdirs-1.4.3-8.11
python3-asn1crypto-0.24.0-9.4
python3-cssselect-1.0.3-21.2
python3-lxml-4.2.1-97.2
python3-mccabe-0.6.1-23.19
python3-py-1.5.2-59.4
python3-pyasn1-0.4.3-83.1
python3-pycodestyle-2.4.0-7.2
python3-pycparser-2.18-18.1
python3-cffi-1.11.5-51.4
python3-pyparsing-2.2.0-1.16
python3-packaging-17.1-21.2
python3-setuptools-39.2.0-128.1
python3-pbr-4.0.3-82.1
python3-linecache2-1.0.0-7.12
python3-traceback2-1.4.0-11.11
python3-pyflakes-1.6.0-50.3
python3-cryptography-2.2.1-106.3
python3-text-unidecode-1.1-5.4
python3-urllib3-1.22-44.2
python3-pycryptodome-3.6.1-13.1
python3-dnspython-1.15.0-54.1
python3-email_validator-1.0.2-3.9
python3-Faker-0.8.10-13.3
python3-pytz-2018.3-74.3
python3-certifi-2018.4.16-32.1
python3-chardet-3.0.4-59.1
python3-numpy-1.14.3-169.5
python3-devel-3.6.5-3.1
python3-Cython-0.28.2-113.2
python3-pandas-0.23.0-50.5
python3-requests-2.18.4-3.35
python3-factory_boy-2.9.2-15.15
python3-pyOpenSSL-17.5.0-2.4
python3-flake8-3.5.0-42.1
python3-unittest2-1.1.0-57.5
python3-mock-2.0.0-58.4
python3-nose-1.3.7-104.4
python3-httpretty-0.8.14-30.9
python3-inflection-0.3.1-3.4
python3-jsondate-0.1.2-5.4
python3-more-itertools-4.1.0-8.1
python3-ndg-httpsclient-0.4.0-6.10
mydata = quandl.get("WIKI/AAPL", rows=5)
is returning the first 5 rows instead of the last 5 rows as it is intended to do.
Sorry, but I have to be frank. I'm sure the code is very good, but the documentation doesn't tell how to use the library. I want the data to get the exchange rate between BTC and LTC. I found this page on the Qualdt site:
https://www.quandl.com/data/BTCE/BTCLTC-BTC-LTC-Exchange-Rate
It tells me to use the code here:
quandl.get("BTCE/BTCLTC", authtoken="my_code")
When install the quandl python library and then paste that code into the shell, I get this:
In [57]: quandl.get("BTCE/BTCLTC", authtoken="my_code")
Out[57]:
High Low Average Last Buy Sell \
Date
2012-07-15 0.005990 0.005264 NaN NaN NaN NaN
2012-07-16 0.006990 0.005519 NaN NaN NaN NaN
2012-07-17 0.007000 0.005091 NaN NaN NaN NaN
2012-07-18 0.006760 0.005000 NaN NaN NaN NaN
2012-07-19 0.005899 0.005000 NaN NaN NaN NaN
2012-07-20 0.007000 0.004900 NaN NaN NaN NaN
2012-07-21 0.006200 0.005550 NaN NaN NaN NaN
2012-07-22 0.006300 0.005556 NaN NaN NaN NaN
2012-07-23 0.006499 0.005601 NaN NaN NaN NaN
2012-07-24 0.006200 0.005244 NaN NaN NaN NaN
2012-07-25 0.005999 0.003503 NaN NaN NaN NaN
2012-07-26 0.005925 0.005050 NaN NaN NaN NaN
The object that is returned seems to be a pandas DataFrame object. How can I iterate through the data found in this dataframe object? The documentation tells me a billion things about everything I don't want to know, but doesn't tell me what I need to know. All I need from the dataset is the average price, and the date.
I tried
[x for x in quandl.get("BTCE/BTCLTC", authtoken="my_code")]
But that doesn't work
I also tried to get the first line in the dataset:
quandl.get("BTCE/BTCLTC", authtoken="my_code")[0]
But that gives me a key error. I then tried this:
quandl.get("BTCE/BTCLTC", authtoken="my_code").as_list()
but that gives me an attribute error
I did try to do
quandl.get("BTCE/BTCLTC", authtoken="my_code").as_matrix()
which does seem to return data, but there is no "Date" field, which makes the data useless for my use.
Why does this have to be so hard? Its very easy to download a csv file, iterate through it, but this quandl library makes it so much harder.
My suggestion is to modify the documentation near the top to explain how to use the most basic usage of this library, and that is downloading a single dataset (or datatable, or database or whatever you call it) and iterate over the data.
Hi. Love the API, but found a possible bug.
From the docs the get function is supposed to support a call such as:
mydata = Quandl.get(["NSE/OIL.4","WIKI/AAPL.1"])
However when I run the following code it returns an empty data frame:
start = '2013-12-01', auth_token = 'my_auth_token'
a_b = q.get(['RAYMOND/FB_REVENUE_A','RAYMOND/FB_NET_INCOME_A'], authtoken=auth_token, trim_start = start)
print( a_b )
Empty DataFrame
Columns: []
Index: []
I confirmed that the Quandl database had the data items in question by running the following code:
a = q.get('RAYMOND/FB_NET_INCOME_A', authtoken=auth_token, trim_start = start)
b = q.get('RAYMOND/FB_REVENUE_A', authtoken=auth_token, trim_start = start)
print( a, b )
Value
Date
2013-12-31 1500000000
Date
2013-12-31 7872000000
I was previously subscribed to the SF0 free dataset, which also provided the ability to bulkdownload the whole dataset. If I understand correctly, the SF0 dataset has now been merged in the SF1 dataset (using the MRY timeframe). Unfortunately SF0 doesn't provide the bulkdownload ability even for the limited data. Is there some mock (or test) dataset, for which the bulkdownload option still works. I need this to verify if my code is working. Thanks in advance.
Hello,
it would be good if there was some way to pass arguments to the requests.get
in the quandl.get
method (or quandl.ApiConfig
class). In particular, the argument verify=False
which disables verification of the SSL certificate in HTTPS: some companies employ MitM in order to record traffic of employees for the compliance purposes, and in this case quandl.get
does not work.
Best
Vladimir
What is the difference betweek .get() and .dataset() ? When I use .get(), it returns all the data on which I can loop, however using .get(), querying for specific date range is not possible which can achieved by .dataset()
.dataset() returns <quandl.model.paginated_list.PaginatedList object at 0x7f4750f77a90> but how to use it?
Don't know if this is expected behavior or not (it wasn't to me).
In short,if you want to get one share:
foo = Quandl.get('GOOG/NYSE_AAPL')
works, but if you want to get one column from one share
foo = Quandl.get('GOOG/NYSE_AAPL.4')
doesn't.
You can get single columns from multiple shares (as on the python demo page):
foo = Quandl.get(['GOOG/NYSE_AAPL.4','GOOG/NASDAQ_MSFT.4'])
And you can also get a column from one share by making it a one-item list:
foo = Quandl.get(['GOOG/NYSE_AAPL.4'])
so it's not a show-stopper, but rather a failure of expectations about the way the "retrieve just one column" thing works.
And while I'm on the subject, Python uses 0-indexed arrays. The fourth column of a Python array bar is
bar[3]
which causes a tiny bit of mental dissonance when I have to type ".4" to fetch the fourth column from the Quandl server in Python.
I try to get all the codes from FRED using:
quandl.get("FRED/codes")
It does not work... however, did I miss anything? Thank you.
Hello,
It will be nice to provide a cache mechanism to Quandl Python.
requests
and requests-cache
could help.
You can see my code here
https://github.com/femtotrader/pandas_datareaders/
Kind regards
Is it possible to have the date column return in the dataframe?
I'm trying to install quandl for python3 but the same error appear in the end:
too much text so I putted in pastebin: http://pastebin.com/VG0hsuBN
The Python client seems to be making an entirely new SSL connection to Quandl for every API call, then throwing it away. This makes multiple requests very slow and causes unnecessary load on Quandl servers. Seems like it would be very easy to store a Session in the Connection object.
http://docs.python-requests.org/en/master/user/advanced/#session-objects
Hi.
I tried to import quandl but I received this Traceback error: "ImportError: No module named 'inflection'" I reviewed your folder and found no file called inflection.py.
This is the message detail:
Traceback (most recent call last):
File "<pyshell#41>", line 1, in
import quandl
File "C:\Python35\lib\site-packages\quandl_init_.py", line 7, in
from .model.database import Database
File "C:\Python35\lib\site-packages\quandl\model\database.py", line 11, in
from quandl.connection import Connection
File "C:\Python35\lib\site-packages\quandl\connection.py", line 5, in
from .util import Util
File "C:\Python\Python35\lib\site-packages\quandl\util.py", line 1, in
from inflection import parameterize
ImportError: No module named 'inflection
Hi,
This looks like a very handy package - thanks!
I've just followed your instructions for multisets using:
data = Quandl.get(['OFDP/ALUMINIUM_21.3',])
but I get a HTTPError: HTTP Error 404: Not Found
. (Adding more series in there does not help.)
Note the URL http://quandl.com/api/v1/multisets.csv?columns=OFDP.ALUMINIUM_21.3,
works ok.
Am I doing something wrong? The example you give in the documentation works fine.
I try install Quandl for python 3 but error appear like this
http://pastebin.com/NYbD6FJh
https://pypi.python.org/pypi/Quandl has version 3.2.0 and no source tarball :(
Plus, there is no commit tagged 3.2.0?
The requests
library is missing in the setup dependencies
EDIT: I just found out that this is only an issue when using earlier versions of pandas (0.8.x in my case).
Running the Quandl module on my local Windows machine went fine. However, I got the following error-message when moving the code to a Debian server:
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python2.7/dist-packages/Quandl/Quandl.py", line 79, in get
urldata = _download(url)
File "/usr/local/lib/python2.7/dist-packages/Quandl/Quandl.py", line 218, in _download
dframe = pd.read_csv(url, index_col=0, parse_dates=True)
File "/usr/lib/pymodules/python2.7/pandas/io/parsers.py", line 236, in read_csv
return _read(TextParser, filepath_or_buffer, kwds)
File "/usr/lib/pymodules/python2.7/pandas/io/parsers.py", line 171, in _read
f = com._get_handle(filepath_or_buffer, 'r', encoding=encoding)
File "/usr/lib/pymodules/python2.7/pandas/core/common.py", line 866, in _get_handle
f = open(path, mode)
This seems to be a URL encoding message as Pandas does not seem to be process UTF-8 encoded URLs which Quandl produces in this case.
Suggested fix:
replace line 220 with
dframe = pd.read_csv(url.encode(sys.getfilesystemencoding()), index_col=0, parse_dates=True)
Firstly, thanks for creating this.
I found out about quandl via https://towardsdatascience.com/time-series-analysis-in-python-an-introduction-70d5a5b1d52a
I'm hoping to reproduce that work but with two different companies. In that the author writes "Quandl does not have number of shares data, but I was able to find average yearly stock shares for both companies with a quick Google search".
Wondering if you are aware of a place to look for average yearly stock shares?
Would it be possible to add params to support the following:
proxies - Requests supports a dictionary containing proxies:
proxies = {'http': 'http://proxy.company.org:8080',
'https': 'http://proxy.company.org:8080',}
verify - requests supports a verify=<file path name of cert,pem>
Would be useful to make requests-kerberos and requests,ntlm work for use behind firewall. This is really useful for the use case where the call to a database redirects to amazon.
Some example code:
import requests
from requests_kerberos import HTTPKerberosAuth
import zipfile, io
proxies = {'http': 'http://proxy.company.org:8080',
'https': 'http://proxy.company.org:8080',}
meta_url = 'https://www.quandl.com/api/v3/datasets/WIKI/AAPL/metadata.json'
# url = 'https://www.quandl.com/api/v3/datasets/WIKI/AAPL.json?collapse=daily?exclude_column_names=true'
url = 'https://www.quandl.com/api/v3/databases/CLSM/data?api_key=<insert key here>
response = requests.get(url, proxies=proxies, verify='cert.pem', auth=HTTPKerberosAuth())
Trying to run the mydata = quandl.get("WIKI/AAPL", rows=5)
example from https://www.quandl.com/tools/python, but running into the following TypeError
:
โ ~ python
Python 2.7.12 (default, Jun 29 2016, 14:05:02)
[GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import quandl
>>> quandl.get("WIKI/AAPL", rows=5)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/site-packages/quandl/get.py", line 48, in get
data = Dataset(dataset_args['code']).data(params=kwargs, handle_column_not_found=True)
File "/usr/local/lib/python2.7/site-packages/quandl/model/dataset.py", line 47, in data
return Data.all(**updated_options)
File "/usr/local/lib/python2.7/site-packages/quandl/operations/list.py", line 17, in all
resource = cls.create_list_from_response(response_data)
File "/usr/local/lib/python2.7/site-packages/quandl/operations/data_list.py", line 13, in create_list_from_response
return DataList(cls, values, metadata)
File "/usr/local/lib/python2.7/site-packages/quandl/model/model_list.py", line 14, in __init__
converted_column_names = Util.convert_column_names(meta)
File "/usr/local/lib/python2.7/site-packages/quandl/util.py", line 102, in convert_column_names
the_list = [Util.methodize(x) for x in meta['column_names']]
File "/usr/local/lib/python2.7/site-packages/quandl/util.py", line 26, in methodize
return parameterize(string.replace(r'.', '')).replace(r'-', '_')
TypeError: an integer is required
The other examples (not involving "WIKI/*"
) seem to be working.
According to PEP8, package names should be lower case.
pip install Quandl
gives "ImportError: No module named pandas"
Downloading/unpacking Quandl
Downloading Quandl-2.2.tar.gz
Running setup.py (path:/home/hobs/.virtualenvs/pugtest/build/Quandl/setup.py) egg_info for package Quandl
Traceback (most recent call last):
File "", line 17, in
File "/home/hobs/.virtualenvs/pugtest/build/Quandl/setup.py", line 6, in
import Quandl
File "Quandl/init.py", line 13, in
from .Quandl import (
File "Quandl/Quandl.py", line 11, in
import pandas as pd
ImportError: No module named pandas
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "", line 17, in
File "/home/hobs/.virtualenvs/pugtest/build/Quandl/setup.py", line 6, in
import Quandl
File "Quandl/init.py", line 13, in
from .Quandl import (
File "Quandl/Quandl.py", line 11, in
import pandas as pd
ImportError: No module named pandas
Can you please add the license file in the root of the repo? This is suggested to include the package on conda-forge.
Thank you.
Hi.
I think there might be a bug in the quandl.get_table when trying to filter the data. Specifically i try to filter the 'ZACKS/EEH' datase using the "obs_date" column to only download recent data. If i try to download the last 20 days the code below works fine, filters correctly, and i can see data up until yesterday. However trying to download for the e.g the last 10 days the code fails with the error:
QuandlError: (Status 500) Something went wrong. Please try again. If you continue to have problems, please contact us at [email protected].
Code example
database = {
'dataset': 'ZACKS/EEH',
'date_filter_column': 'obs_date'
}
import quandl
import datetime
n_days_back = 10 #change to e.g. 20 to make the code work
today = datetime.date.today()
start_delta = datetime.timedelta(days=n_days_back)
start_of_week = today - start_delta
start_date = "%d-%0.2d-%0.2d"%(start_of_week.year, start_of_week.month, start_of_week.day)
print(start_date)
options = {'datatable_code': database['dataset'],
'paginate': True,
database['date_filter_column']: {'gte': start_date}}
data = quandl.get_table(**options)
Traceback
QuandlError Traceback (most recent call last)
<ipython-input-149-cc398ea52228> in <module>()
20 'paginate': True,
21 database['date_filter_column']: {'gte': start_date}}
---> 22 data = quandl.get_table(**options)
/Users/casperkaae/anaconda3/lib/python3.6/site-packages/quandl/get_table.py in get_table(datatable_code, **options)
17 while True:
18 next_options = copy.deepcopy(options)
---> 19 next_data = Datatable(datatable_code).data(params=next_options)
20
21 if data is None:
/Users/casperkaae/anaconda3/lib/python3.6/site-packages/quandl/model/datatable.py in data(self, **options)
14 def data(self, **options):
15 updated_options = Util.convert_options(**options)
---> 16 return Data.page(self, **updated_options)
/Users/casperkaae/anaconda3/lib/python3.6/site-packages/quandl/operations/list.py in page(cls, datatable, **options)
22 params = {'id': str(datatable.code)}
23 path = Util.constructed_path(datatable.default_path(), params)
---> 24 r = Connection.request('get', path, **options)
25 response_data = r.json()
26 Util.convert_to_dates(response_data)
/Users/casperkaae/anaconda3/lib/python3.6/site-packages/quandl/connection.py in request(cls, http_verb, url, **options)
34 abs_url = '%s/%s' % (ApiConfig.api_base, url)
35
---> 36 return cls.execute_request(http_verb, abs_url, **options)
37
38 @classmethod
/Users/casperkaae/anaconda3/lib/python3.6/site-packages/quandl/connection.py in execute_request(cls, http_verb, url, **options)
42 response = func(url, **options)
43 if response.status_code < 200 or response.status_code >= 300:
---> 44 cls.handle_api_error(response)
45 else:
46 return response
/Users/casperkaae/anaconda3/lib/python3.6/site-packages/quandl/connection.py in handle_api_error(cls, resp)
64 # throw generic error
65 if 'quandl_error' not in error_body:
---> 66 raise QuandlError(http_status=resp.status_code, http_body=resp.text)
67
68 code = error_body['quandl_error']['code']
QuandlError: (Status 500) Something went wrong. Please try again. If you continue to have problems, please contact us at connect@quandl.com
This library could use some tests.
I'm happy to work on this (and have started on it, using unittest
) but I'm wondering what the best way to go about doing this might be? I have a few questions:
Is there a particular dataset I should test against? To avoid flooding Quandl with test datasets (needed to test the "push" functionality), is there a way to programmatically delete datasets that are associated with an account/api key? Should there be a API key just for running tests? (Probably not, but otherwise how should we run them?)
Is anyone aware of any examples of good libraries wrapping around a service like Quandl (in whichever language, though Python would be ideal)? I'd love to learn.
We're trying to use google-cloud-python
to pull data from GCS, but requests
< 2.18 causes an error.
Hello,
import Quandl
Quandl.get(["CHRIS/CME_ES1", "CHRIS/CME_NQ1"])
returns a DataFrame
Isn't there a way to get a Pandas Panel or an xarray.DataArray
An other approach could be to simply return an ordered dict with DataFrame as "value" and ticker as key.
User will be responsible for converting to 3D datastructure.
Kind regards
pip wants to find MSVC++ - I have the complied version of Numpy installed and working (i'm trying to upgrade from 2.89)
Currently requirements.txt only allows for numpy 1.7.1 and pandas 0.11. Users with newer versions are therefore forced to install older versions of these modules. Requirements.txt should be changed from:
numpy==1.7.1
pandas==0.11
to:
numpy>=1.7.1
pandas>=0.11
This saves users from having to recompile an old version of numpy, if they already have a newer version installed.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.