Giter VIP home page Giter VIP logo

quandl-python's People

Contributors

42b avatar a-scott-rowe avatar brianrisk avatar ccleung avatar chrisstevens avatar couture-ql avatar davidkimdy avatar edschofield avatar ericvautour-quandl avatar fengshuo avatar gaiusmarius avatar jbyck avatar jjmar avatar jlovison avatar jlowin avatar kongscn avatar mbasset avatar mikkel14 avatar mmaker avatar oliversherouse avatar sebix avatar snehpatel3 avatar somethinrother avatar tiagocordeiro avatar ulf1 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

quandl-python's Issues

ImportError: No module named 'inflection

Hi.

I tried to import quandl but I received this Traceback error: "ImportError: No module named 'inflection'" I reviewed your folder and found no file called inflection.py.

This is the message detail:

Traceback (most recent call last):
File "<pyshell#41>", line 1, in
import quandl
File "C:\Python35\lib\site-packages\quandl_init_.py", line 7, in
from .model.database import Database
File "C:\Python35\lib\site-packages\quandl\model\database.py", line 11, in
from quandl.connection import Connection
File "C:\Python35\lib\site-packages\quandl\connection.py", line 5, in
from .util import Util
File "C:\Python\Python35\lib\site-packages\quandl\util.py", line 1, in
from inflection import parameterize
ImportError: No module named 'inflection

Issue - 1

Hi

I came across this issue while trying to install quandl (python):

x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c build/temp.linux-x86_64-2.7/_openssl.c -o build/temp.linux-x86_64-2.7/build/temp.linux-x86_64-2.7/_openssl.o
build/temp.linux-x86_64-2.7/_openssl.c:12:24: fatal error: pyconfig.h: No such file or directory
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

----------------------------------------

Command "/usr/bin/python -u -c "import setuptools, tokenize;file='/tmp/pip-build-1wgahm/cryptography/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-RV3Adm-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-1wgahm/cryptography/

unneeded ipython dependency

Is the dependency on ipython ("from IPython import embed") needed ? it does not look used anywhere in the code.

Quandl not installing in venv

I tried to install in a virtual env a but quandl is getting installed in global env
Please rectify this problem

Sorry you can kindly close this it was a set up issue in my venv

store and read api key from local file

Current Way

import quandl
quandl.ApiConfig.api_key = "mysecretkey"

Store a API Key

Save API Key string in a local text file, e.g. $HOME/.quandl_apikey

quandl.apikey_set("mysecretkey")

or a specified file, e.g.

quandl.apikey_set("mysecretkey", filepath="/data/.quandl_apikey")

Use cases

  • Users who don't want to expose their API key when sharing a Jupyter Notebook
  • Docker container with mount point /data/ to external storage (It's really painful to inject it with run command, and storing keys and tokens inside container is bad practice)

Implemention

https://github.com/ulf1/updoon/blob/master/updoon/quandl_apikey.py

this would read the file with the api like

import quandl
import updoon 
quandl.ApiConfig.api_key = updoon.quandl_apikey()

Proposal

Add save_key and read_key to api_config.py, e.g.

class ApiConfig:
     ....

def read_key(filepath=None):
    ...
    ApiConfig.api_key = ...

to do

import quandl
quandl.read_key()
...

instead of exposing the API key within a script.

conda install fetch 2.8.9 version

Hello,

conda install fetch 2.8.9 version

$ conda --version
conda 4.1.11
$ python --version
Python 3.5.2 :: Anaconda custom (x86_64)

$ conda install quandl
Fetching package metadata .......
Solving package specifications: ..........

Package plan for installation in environment /Users/femto/anaconda:

The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    quandl-2.8.9               |           py35_0          11 KB

The following NEW packages will be INSTALLED:

    quandl: 2.8.9-py35_0

Proceed ([y]/n)? y

Fetching packages ...
quandl-2.8.9-p 100% |############################################################################################################################################################| Time: 0:00:00 787.71 kB/s
Extracting packages ...
[      COMPLETE      ]|###############################################################################################################################################################################| 100%
Linking packages ...
[      COMPLETE      ]|###############################################################################################################################################################################| 100%
$ ipython
Python 3.5.2 |Anaconda custom (x86_64)| (default, Jul  2 2016, 17:52:12)
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import quandl
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-ab7fa5ad38a5> in <module>()
----> 1 import quandl

ImportError: No module named 'quandl'

In [2]: import Quandl

In [3]: Quandl.__version__
Out[3]: '2.8.9'

In [4]: import pandas as pd

In [5]: pd.set_option("max_rows", 10)

In [6]: Quandl.get("ISM/MAN_PMI")
Out[6]:
            Index
Date
1948-01-01   51.7
1948-02-01   50.2
1948-03-01   43.3
1948-04-01   45.4
1948-05-01   49.5
...           ...
2016-03-01   51.8
2016-04-01   50.8
2016-05-01   51.3
2016-06-01   53.2
2016-07-01   52.6

[823 rows x 1 columns]

I thought that current version was 3.

Kind regards

pyasn1 distro installation fails for Quandl 3.0.1

Hi All,

I get the following error on installing Quandl 3.0.1 for Python 2.7.

Processing dependencies for Quandl==3.0.1
Searching for pyasn1
Reading https://pypi.python.org/simple/pyasn1/
Download error on https://pypi.python.org/simple/pyasn1/: [Errno 11004] getaddri
nfo failed -- Some packages may not be found!
Couldn't find index page for 'pyasn1' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading https://pypi.python.org/simple/
Download error on https://pypi.python.org/simple/: [Errno 11004] getaddrinfo fai
led -- Some packages may not be found!
No local packages or download links found for pyasn1
error: Could not find suitable distribution for Requirement.parse('pyasn1')

Please suggest if there is an alternative.

Thanks,
Best,
Pruthvi

European Commission Business and Consumer Surveys

I'm not sure if this issue has been raised before elsewhere.

I was curious to learn why some of the free Quandl economic databases often update without the most recent data. For example, the European Commission Business and Consumer Surveys Quandl dataset doesn't currently include the most recent data (February, 2018). It looks like Quandl is updating on the correct release date, but the new data isn't available until subsequent updates.

I was also having similar issues with the Institute for Supply Management in January, however, this no longer seems to be an issue.

Thanks in advance.

`ImportError: No module named pathlib` and wrong exception

# 1

#117 introduced a bug in regards to Python 2 compatibility since pathlib is Python 3.4+ only.

Python 2.7.15rc1 (default, Nov 12 2018, 14:31:15) 
[GCC 7.3.0] on linux2
>>> import quandl
>>> quandl.read_key()
Traceback (most recent call last):
...
ImportError: No module named pathlib
>>> quandl.save_key('secret...')
Traceback (most recent call last):
...
ImportError: No module named pathlib

Currently, the only way to hit these import statements is to not pass a filename argument to either function (or pass filename=None). Here are the two sections causing the issue:

def save_key(apikey, filename=None):
if filename is None:
import pathlib
filename = str(pathlib.Path.home()) + "/.quandl_apikey"

def read_key(filename=None):
if filename is None:
import pathlib
filename = str(pathlib.Path.home()) + "/.quandl_apikey"

This problem would likely have been caught earlier if the imports were at the top of the file as mentioned in PEP8.

from .api_config import ApiConfig, save_key, read_key

A simple fix seems to be indicated in the Python docs themselves:

Return a new path object representing the userโ€™s home directory (as returned by os.path.expanduser() with ~ construct)

Python 3.7.1 (default, Oct 25 2018, 21:25:21) 
[GCC 7.3.0] on linux
>>> import os
>>> import pathlib
>>> os.path.expanduser('~') == str(pathlib.Path.home())
True

There is a problem with simply replacing str(pathlib.Path.home()) with os.path.expanduser('~') in these two lines though. The string concatenation is prepending forward slash in front of .quandl_apikey which (if I remember correctly) is not compatible with Windows which uses backslashes.

filename = str(pathlib.Path.home()) + "/.quandl_apikey" 
                                       ^

I'm a fan of pathlib and one way to solve this would be like this:

>>> pathlib.Path.home() / ".quandl_apikey"
PosixPath('/home/.../.quandl_apikey')

But since Python 2 compatibility is the issue, using os.path.join seems like an appropriate fix.

>>> p = pathlib.Path.home() / ".quandl_apikey"
>>> str(p) == os.path.join(os.path.expanduser('~'), '.quandl_apikey')
True

# 2

There is also a smaller bug where the try/except in read_key is looking for the wrong exception. In Python 2, open raises an IOError if a file isn't found. In Python 3, an OSError is raised.

Neither of these exceptions have a parent exception of ValueError though.

Python 2.7.15rc1 (default, Nov 12 2018, 14:31:15) 
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> try:
...     with open('doesnt_exist.txt', 'r') as f:
...         pass
... except ValueError:
...     raise Exception("File '{:s}' not found.".format(filename))
... 
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
IOError: [Errno 2] No such file or directory: 'doesnt_exist.txt'

Python 3.7.1 (default, Oct 25 2018, 21:25:21) 
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> try:
...     with open('doesnt_exist.txt', 'r') as f:
...         pass
... except ValueError:
...     raise Exception("File '{:s}' not found.".format(filename))
... 
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
FileNotFoundError: [Errno 2] No such file or directory: 'doesnt_exist.txt'

In my opinion, these exceptions are already very clear about what went wrong and it seems unnecessary to catch them in read_key just to simply re-raise them with a custom message.


tl;dr

Ignoring the fact that these two functions are mutating the state of a class and seem like they could be class methods instead... maybe something like this?

import os


class ApiConfig:
    api_key = None
    api_protocol = 'https://'
    api_base = '{}www.quandl.com/api/v3'.format(api_protocol)
    api_version = None
    page_limit = 100

    use_retries = True
    number_of_retries = 5
    retry_backoff_factor = 0.5
    max_wait_between_retries = 8
    retry_status_codes = [429] + list(range(500, 512))


def save_key(apikey, filename=None):
    if filename is None:
        filename = os.path.join(os.path.expanduser('~'), '.quandl_apikey')

    with open(filename, 'w') as f:
        f.write(apikey)

    ApiConfig.api_key = apikey


def read_key(filename=None):
    if filename is None:
        filename = os.path.join(os.path.expanduser('~'), '.quandl_apikey')

    with open(filename, 'r') as f:
        apikey = f.read()

    if not apikey:
        raise ValueError("File '{:s}' is empty.".format(filename))

    ApiConfig.api_key = apikey

Http 403 error with correct api key

data = quandl.get("FRED/GDP")
print(data.head())
With correct api key (already renewed api key and checked with curl the same request as https://www.quandl.com/api/v3/datasets/FRED/GDP/data.json?api_key=XXXXXXXXXX and it returns correct formatted json) but Quandl 3.4.3 does return:


JSONDecodeError Traceback (most recent call last)
/usr/local/lib/python3.5/dist-packages/quandl/connection.py in parse(cls, response)
54 try:
---> 55 return response.json()
56 except ValueError:

/usr/local/lib/python3.5/dist-packages/requests/models.py in json(self, **kwargs)
895 pass
--> 896 return complexjson.loads(self.text, **kwargs)
897

/usr/lib/python3.5/json/init.py in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
318 parse_constant is None and object_pairs_hook is None and not kw):
--> 319 return _default_decoder.decode(s)
320 if cls is None:

/usr/lib/python3.5/json/decoder.py in decode(self, s, _w)
338 """
--> 339 obj, end = self.raw_decode(s, idx=_w(s, 0).end())
340 end = _w(s, end).end()

/usr/lib/python3.5/json/decoder.py in raw_decode(self, s, idx)
356 except StopIteration as err:
--> 357 raise JSONDecodeError("Expecting value", s, err.value) from None
358 return obj, end

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

QuandlError Traceback (most recent call last)
in
----> 1 data = quandl.get("FRED/GDP")
2 print(data.head())

/usr/local/lib/python3.5/dist-packages/quandl/get.py in get(dataset, **kwargs)
46 if dataset_args['column_index'] is not None:
47 kwargs.update({'column_index': dataset_args['column_index']})
---> 48 data = Dataset(dataset_args['code']).data(params=kwargs, handle_column_not_found=True)
49 # Array
50 elif isinstance(dataset, list):

/usr/local/lib/python3.5/dist-packages/quandl/model/dataset.py in data(self, **options)
45 updated_options = Util.merge_options('params', params, **options)
46 try:
---> 47 return Data.all(**updated_options)
48 except NotFoundError:
49 if handle_not_found_error:

/usr/local/lib/python3.5/dist-packages/quandl/operations/list.py in all(cls, **options)
12 options['params'] = {}
13 path = Util.constructed_path(cls.list_path(), options['params'])
---> 14 r = Connection.request('get', path, **options)
15 response_data = r.json()
16 Util.convert_to_dates(response_data)

/usr/local/lib/python3.5/dist-packages/quandl/connection.py in request(cls, http_verb, url, **options)
34 abs_url = '%s/%s' % (ApiConfig.api_base, url)
35
---> 36 return cls.execute_request(http_verb, abs_url, **options)
37
38 @classmethod

/usr/local/lib/python3.5/dist-packages/quandl/connection.py in execute_request(cls, http_verb, url, **options)
42 response = func(url, **options)
43 if response.status_code < 200 or response.status_code >= 300:
---> 44 cls.handle_api_error(response)
45 else:
46 return response

/usr/local/lib/python3.5/dist-packages/quandl/connection.py in handle_api_error(cls, resp)
59 @classmethod
60 def handle_api_error(cls, resp):
---> 61 error_body = cls.parse(resp)
62
63 # if our app does not form a proper quandl_error response

/usr/local/lib/python3.5/dist-packages/quandl/connection.py in parse(cls, response)
55 return response.json()
56 except ValueError:
---> 57 raise QuandlError(http_status=response.status_code, http_body=response.text)
58
59 @classmethod

QuandlError: (Status 403) Something went wrong. Please try again. If you continue to have problems, please contact us at [email protected].

Package resolution warning in conda update

I'm getting the following warning on issuing the following conda command:

conda update --all
Fetching package metadata .............
Solving package specifications: .
Warning: 2 possible package resolutions (only showing differing packages):
  - defaults::quandl-3.2.0-py36h30f7a5e_0, defaults::requests-2.14.2-py36_0
  - defaults::quandl-3.1.0-py36_0, defaults::requests-2.18.4-py36h4371aae_1

Here's my conda info:

Current conda install:

               platform : win-64
          conda version : 4.3.30
       conda is private : False
      conda-env version : 4.3.30
    conda-build version : 3.0.30
         python version : 3.6.3.final.0
       requests version : 2.14.2
       root environment : D:\Users\803375527\AppData\Local\Continuum\anaconda3  (writable)
    default environment : D:\Users\803375527\AppData\Local\Continuum\anaconda3
       envs directories : D:\Users\803375527\AppData\Local\Continuum\anaconda3\envs
                          C:\Users\803375527\AppData\Local\conda\conda\envs
                          C:\Users\803375527\.conda\envs
          package cache : D:\Users\803375527\AppData\Local\Continuum\anaconda3\pkgs
                          C:\Users\803375527\AppData\Local\conda\conda\pkgs
           channel URLs : https://repo.continuum.io/pkgs/main/win-64
                          https://repo.continuum.io/pkgs/main/noarch
                          https://repo.continuum.io/pkgs/free/win-64
                          https://repo.continuum.io/pkgs/free/noarch
                          https://repo.continuum.io/pkgs/r/win-64
                          https://repo.continuum.io/pkgs/r/noarch
                          https://repo.continuum.io/pkgs/pro/win-64
                          https://repo.continuum.io/pkgs/pro/noarch
                          https://repo.continuum.io/pkgs/msys2/win-64
                          https://repo.continuum.io/pkgs/msys2/noarch
            config file : C:\Users\803375527\.condarc
             netrc file : None
           offline mode : False
             user-agent : conda/4.3.30 requests/2.14.2 CPython/3.6.3 Windows/10 Windows/10.0.14393
          administrator : False

Splits are not showing up in adjusted data from WIKI

In fact, adjusted data is identical to non-adjusted data. Consider this example (stock = CGNX):
Here is output from 2017-11-20 to 2017-12-18. https://imgur.com/a/PAd4s

This is data pulled like so:
test = quandl.get('WIKI/CGNX')

Split ratio remains at 1.0 throughout, and adjusted & unadjusted values are identical. However there is ample evidence of a stock split around the end of October, and any properly adjusted data shows this:
https://imgur.com/a/qGUfw (from Yahoo Finance, all values below $100).

Received "Quandl Error QECx02" when the argument of quandl.get() has special characters

I am seeing

incorrect Quandl code error

when the argument of quandl.get() function has special characters.

code snippet:

import quandl
mydata = quandl.get("NSE/BAJAJ-AUTO")

Issue:

quandl.errors.quandl_error.NotFoundError: (Status 404) (Quandl Error QECx02) You have submitted an incorrect Quandl code. Please check your Quandl codes and try again.

if/elif/else could be replaced by a dict

Hello,

I'm reading you code...

if I where you I won't write in https://github.com/quandl/quandl-python/blob/feature-AP-243/v3-client/quandl/connection.py

if code_letter == 'L':
    klass = LimitExceededError
elif code_letter == 'M':
    klass = InternalServerError
elif code_letter == 'A':
    klass = AuthenticationError
elif code_letter == 'P':
    klass = ForbiddenError
elif code_letter == 'S':
    klass = InvalidRequestError
elif code_letter == 'C':
    klass = NotFoundError
elif code_letter == 'X':
    klass = ServiceUnavailableError
else:
    klass = QuandlError

but

d_klass = {
    'L': LimitExceededError,
    'M': InternalServerError,
    'A': AuthenticationError,
    'P': ForbiddenError,
    'S': InvalidRequestError,
    'C': NotFoundError,
    'X': ServiceUnavailableError
}
klass_other = QuandlError
try:
    klass = d_klass[code_letter]
except KeyError:
    klass = QuandlError

Kind regards

Possible Pandas Initialization Problem

If I download data using this Quandl.get() method and then subsequently plot the data the date range is inverted. This does not happen if I initialize a pandas.core.series.TimeSeries myself.

%pylab inline
import Quandl
import pandas as pd

tsx = Quandl.get("YAHOO/INDEX_GSPTSE")
print type(tsx['Open'])
print tsx['Open']
tsx['Open'].plot()


ds = pd.date_range(start='2012-01-01', periods=100)[::-1]
x = pd.Series(range(100), index=ds)
print type(x)
x.head()
x.plot()

build fail at test_get_merged_dataset_data_to_list

Since some days, the builds are failing at openSUSES build system with this error:

[   32s] ======================================================================
[   32s] FAIL: test_get_merged_dataset_data_to_list (test.test_merged_dataset.GetMergedDatasetTest)
[   32s] ----------------------------------------------------------------------
[   32s] Traceback (most recent call last):
[   32s]   File "/home/abuild/rpmbuild/BUILD/quandl-python-3.3.0/test/test_merged_dataset.py", line 206, in test_get_merged_dataset_data_to_list
[   32s]     self.assertItemsEqual(results[index], expected_item)
[   32s] AssertionError: Expected, but missing:
[   32s]     [1436572800000000000L]
[   32s] Unexpected, but present:
[   32s]     [datetime.datetime(2015, 7, 11, 0, 0)]
[   32s] -------------------- >> begin captured logging << --------------------
[   32s] urllib3.connectionpool: DEBUG: Starting new HTTPS connection (1): www.quandl.com
[   32s] urllib3.connectionpool: DEBUG: https://www.quandl.com:443 "GET /api/v3/datasets/NSE/OIL/data?order=asc HTTP/1.1" 200 276
[   32s] urllib3.connectionpool: DEBUG: Starting new HTTPS connection (1): www.quandl.com
[   32s] urllib3.connectionpool: DEBUG: https://www.quandl.com:443 "GET /api/v3/datasets/SINGLE/COLUMN/data?column_index=1&order=asc HTTP/1.1" 200 227
[   32s] urllib3.connectionpool: DEBUG: Starting new HTTPS connection (1): www.quandl.com
[   32s] urllib3.connectionpool: DEBUG: https://www.quandl.com:443 "GET /api/v3/datasets/WIKI/MSFT/data?order=asc HTTP/1.1" 200 276
[   32s] --------------------- >> end captured logging << ---------------------
[   32s] 
[   32s] ----------------------------------------------------------------------

Full log is here: https://build.opensuse.org/build/devel:languages:python/openSUSE_Tumbleweed/x86_64/python-Quandl/_log

I suspect an incompatibility with some dependency (probably dateutil?), these are the ones used:

python3-base-3.6.5-3.1
python3-six-1.11.0-66.7
python3-idna-2.6-10.5
python3-3.6.5-3.1
python3-python-dateutil-2.6.1-14.14
python3-appdirs-1.4.3-8.11
python3-asn1crypto-0.24.0-9.4
python3-cssselect-1.0.3-21.2
python3-lxml-4.2.1-97.2
python3-mccabe-0.6.1-23.19
python3-py-1.5.2-59.4
python3-pyasn1-0.4.3-83.1
python3-pycodestyle-2.4.0-7.2
python3-pycparser-2.18-18.1
python3-cffi-1.11.5-51.4
python3-pyparsing-2.2.0-1.16
python3-packaging-17.1-21.2
python3-setuptools-39.2.0-128.1
python3-pbr-4.0.3-82.1
python3-linecache2-1.0.0-7.12
python3-traceback2-1.4.0-11.11
python3-pyflakes-1.6.0-50.3
python3-cryptography-2.2.1-106.3
python3-text-unidecode-1.1-5.4
python3-urllib3-1.22-44.2
python3-pycryptodome-3.6.1-13.1
python3-dnspython-1.15.0-54.1
python3-email_validator-1.0.2-3.9
python3-Faker-0.8.10-13.3
python3-pytz-2018.3-74.3
python3-certifi-2018.4.16-32.1
python3-chardet-3.0.4-59.1
python3-numpy-1.14.3-169.5
python3-devel-3.6.5-3.1
python3-Cython-0.28.2-113.2
python3-pandas-0.23.0-50.5
python3-requests-2.18.4-3.35
python3-factory_boy-2.9.2-15.15
python3-pyOpenSSL-17.5.0-2.4
python3-flake8-3.5.0-42.1
python3-unittest2-1.1.0-57.5
python3-mock-2.0.0-58.4
python3-nose-1.3.7-104.4
python3-httpretty-0.8.14-30.9
python3-inflection-0.3.1-3.4
python3-jsondate-0.1.2-5.4
python3-more-itertools-4.1.0-8.1
python3-ndg-httpsclient-0.4.0-6.10

Documentation is absolutely garbage

Sorry, but I have to be frank. I'm sure the code is very good, but the documentation doesn't tell how to use the library. I want the data to get the exchange rate between BTC and LTC. I found this page on the Qualdt site:

https://www.quandl.com/data/BTCE/BTCLTC-BTC-LTC-Exchange-Rate

It tells me to use the code here:

quandl.get("BTCE/BTCLTC", authtoken="my_code")

When install the quandl python library and then paste that code into the shell, I get this:

In [57]: quandl.get("BTCE/BTCLTC", authtoken="my_code")
Out[57]:
High       Low   Average     Last      Buy     Sell  \
Date
2012-07-15  0.005990  0.005264       NaN      NaN      NaN      NaN
2012-07-16  0.006990  0.005519       NaN      NaN      NaN      NaN
2012-07-17  0.007000  0.005091       NaN      NaN      NaN      NaN
2012-07-18  0.006760  0.005000       NaN      NaN      NaN      NaN
2012-07-19  0.005899  0.005000       NaN      NaN      NaN      NaN
2012-07-20  0.007000  0.004900       NaN      NaN      NaN      NaN
2012-07-21  0.006200  0.005550       NaN      NaN      NaN      NaN
2012-07-22  0.006300  0.005556       NaN      NaN      NaN      NaN
2012-07-23  0.006499  0.005601       NaN      NaN      NaN      NaN
2012-07-24  0.006200  0.005244       NaN      NaN      NaN      NaN
2012-07-25  0.005999  0.003503       NaN      NaN      NaN      NaN
2012-07-26  0.005925  0.005050       NaN      NaN      NaN      NaN

The object that is returned seems to be a pandas DataFrame object. How can I iterate through the data found in this dataframe object? The documentation tells me a billion things about everything I don't want to know, but doesn't tell me what I need to know. All I need from the dataset is the average price, and the date.

I tried

[x for x in quandl.get("BTCE/BTCLTC", authtoken="my_code")]

But that doesn't work

I also tried to get the first line in the dataset:

quandl.get("BTCE/BTCLTC", authtoken="my_code")[0]

But that gives me a key error. I then tried this:

quandl.get("BTCE/BTCLTC", authtoken="my_code").as_list()

but that gives me an attribute error

I did try to do

quandl.get("BTCE/BTCLTC", authtoken="my_code").as_matrix()

which does seem to return data, but there is no "Date" field, which makes the data useless for my use.

Why does this have to be so hard? Its very easy to download a csv file, iterate through it, but this quandl library makes it so much harder.

My suggestion is to modify the documentation near the top to explain how to use the most basic usage of this library, and that is downloading a single dataset (or datatable, or database or whatever you call it) and iterate over the data.

Quandl's get function doesn't support multiset calls to SEC data

Hi. Love the API, but found a possible bug.

From the docs the get function is supposed to support a call such as:
mydata = Quandl.get(["NSE/OIL.4","WIKI/AAPL.1"])

However when I run the following code it returns an empty data frame:
start = '2013-12-01', auth_token = 'my_auth_token'

a_b = q.get(['RAYMOND/FB_REVENUE_A','RAYMOND/FB_NET_INCOME_A'], authtoken=auth_token, trim_start = start)
print( a_b )
Empty DataFrame
Columns: []
Index: []

I confirmed that the Quandl database had the data items in question by running the following code:

a = q.get('RAYMOND/FB_NET_INCOME_A', authtoken=auth_token, trim_start = start)
b = q.get('RAYMOND/FB_REVENUE_A', authtoken=auth_token, trim_start = start)
print( a, b )
Value
Date
2013-12-31 1500000000
Date
2013-12-31 7872000000

bulkdownload testing

I was previously subscribed to the SF0 free dataset, which also provided the ability to bulkdownload the whole dataset. If I understand correctly, the SF0 dataset has now been merged in the SF1 dataset (using the MRY timeframe). Unfortunately SF0 doesn't provide the bulkdownload ability even for the limited data. Is there some mock (or test) dataset, for which the bulkdownload option still works. I need this to verify if my code is working. Thanks in advance.

Passing `requests` arguments through `get`

Hello,

it would be good if there was some way to pass arguments to the requests.get in the quandl.get method (or quandl.ApiConfig class). In particular, the argument verify=False which disables verification of the SSL certificate in HTTPS: some companies employ MitM in order to record traffic of employees for the compliance purposes, and in this case quandl.get does not work.

Best
Vladimir

How to use .dataset() result ?

What is the difference betweek .get() and .dataset() ? When I use .get(), it returns all the data on which I can loop, however using .get(), querying for specific date range is not possible which can achieved by .dataset()
.dataset() returns <quandl.model.paginated_list.PaginatedList object at 0x7f4750f77a90> but how to use it?

Subsetting by columns fails with one share

Don't know if this is expected behavior or not (it wasn't to me).

In short,if you want to get one share:

foo = Quandl.get('GOOG/NYSE_AAPL') 

works, but if you want to get one column from one share

foo = Quandl.get('GOOG/NYSE_AAPL.4')

doesn't.

You can get single columns from multiple shares (as on the python demo page):

 foo = Quandl.get(['GOOG/NYSE_AAPL.4','GOOG/NASDAQ_MSFT.4']) 

And you can also get a column from one share by making it a one-item list:

foo = Quandl.get(['GOOG/NYSE_AAPL.4'])

so it's not a show-stopper, but rather a failure of expectations about the way the "retrieve just one column" thing works.


And while I'm on the subject, Python uses 0-indexed arrays. The fourth column of a Python array bar is

bar[3]

which causes a tiny bit of mental dissonance when I have to type ".4" to fetch the fourth column from the Quandl server in Python.

ImportError: No module named 'inflection'

Hi.

I tried to import quandl but I received this Traceback error: "ImportError: No module named 'inflection'" I reviewed your folder and found no file called inflection.py.

This is the message detail:

Traceback (most recent call last):
File "<pyshell#41>", line 1, in
import quandl
File "C:\Python35\lib\site-packages\quandl_init_.py", line 7, in
from .model.database import Database
File "C:\Python35\lib\site-packages\quandl\model\database.py", line 11, in
from quandl.connection import Connection
File "C:\Python35\lib\site-packages\quandl\connection.py", line 5, in
from .util import Util
File "C:\Python\Python35\lib\site-packages\quandl\util.py", line 1, in
from inflection import parameterize
ImportError: No module named 'inflection

Problem with multiseries

Hi,
This looks like a very handy package - thanks!
I've just followed your instructions for multisets using:

data = Quandl.get(['OFDP/ALUMINIUM_21.3',])

but I get a HTTPError: HTTP Error 404: Not Found. (Adding more series in there does not help.)

Note the URL http://quandl.com/api/v1/multisets.csv?columns=OFDP.ALUMINIUM_21.3, works ok.

Am I doing something wrong? The example you give in the documentation works fine.

URL UTF-8 encoding issue

EDIT: I just found out that this is only an issue when using earlier versions of pandas (0.8.x in my case).

Running the Quandl module on my local Windows machine went fine. However, I got the following error-message when moving the code to a Debian server:

Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python2.7/dist-packages/Quandl/Quandl.py", line 79, in get
urldata = _download(url)
File "/usr/local/lib/python2.7/dist-packages/Quandl/Quandl.py", line 218, in _download
dframe = pd.read_csv(url, index_col=0, parse_dates=True)
File "/usr/lib/pymodules/python2.7/pandas/io/parsers.py", line 236, in read_csv
return _read(TextParser, filepath_or_buffer, kwds)
File "/usr/lib/pymodules/python2.7/pandas/io/parsers.py", line 171, in _read
f = com._get_handle(filepath_or_buffer, 'r', encoding=encoding)
File "/usr/lib/pymodules/python2.7/pandas/core/common.py", line 866, in _get_handle
f = open(path, mode)

This seems to be a URL encoding message as Pandas does not seem to be process UTF-8 encoded URLs which Quandl produces in this case.

Suggested fix:
replace line 220 with
dframe = pd.read_csv(url.encode(sys.getfilesystemencoding()), index_col=0, parse_dates=True)

looking for yearly stock shares

Firstly, thanks for creating this.

I found out about quandl via https://towardsdatascience.com/time-series-analysis-in-python-an-introduction-70d5a5b1d52a

I'm hoping to reproduce that work but with two different companies. In that the author writes "Quandl does not have number of shares data, but I was able to find average yearly stock shares for both companies with a quick Google search".

Wondering if you are aware of a place to look for average yearly stock shares?

Quandl-Python add params

Would it be possible to add params to support the following:

proxies - Requests supports a dictionary containing proxies:

proxies = {'http': 'http://proxy.company.org:8080',
       'https': 'http://proxy.company.org:8080',}

verify - requests supports a verify=<file path name of cert,pem>

Would be useful to make requests-kerberos and requests,ntlm work for use behind firewall. This is really useful for the use case where the call to a database redirects to amazon.

Some example code:

import requests
from requests_kerberos import HTTPKerberosAuth
import zipfile, io

proxies = {'http': 'http://proxy.company.org:8080',
       'https': 'http://proxy.company.org:8080',}

meta_url = 'https://www.quandl.com/api/v3/datasets/WIKI/AAPL/metadata.json'

# url =  'https://www.quandl.com/api/v3/datasets/WIKI/AAPL.json?collapse=daily?exclude_column_names=true'

url =  'https://www.quandl.com/api/v3/databases/CLSM/data?api_key=<insert key here>

response = requests.get(url, proxies=proxies, verify='cert.pem', auth=HTTPKerberosAuth())

TypeError on API example

Environment

  • macOS 10.12.4
  • Issue occurs on Python 2.7.12 and 3.6.0
  • quandl version 3.1.0

Issue

Trying to run the mydata = quandl.get("WIKI/AAPL", rows=5) example from https://www.quandl.com/tools/python, but running into the following TypeError:

Example

โžœ  ~ python
Python 2.7.12 (default, Jun 29 2016, 14:05:02)
[GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import quandl
>>> quandl.get("WIKI/AAPL", rows=5)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/site-packages/quandl/get.py", line 48, in get
    data = Dataset(dataset_args['code']).data(params=kwargs, handle_column_not_found=True)
  File "/usr/local/lib/python2.7/site-packages/quandl/model/dataset.py", line 47, in data
    return Data.all(**updated_options)
  File "/usr/local/lib/python2.7/site-packages/quandl/operations/list.py", line 17, in all
    resource = cls.create_list_from_response(response_data)
  File "/usr/local/lib/python2.7/site-packages/quandl/operations/data_list.py", line 13, in create_list_from_response
    return DataList(cls, values, metadata)
  File "/usr/local/lib/python2.7/site-packages/quandl/model/model_list.py", line 14, in __init__
    converted_column_names = Util.convert_column_names(meta)
  File "/usr/local/lib/python2.7/site-packages/quandl/util.py", line 102, in convert_column_names
    the_list = [Util.methodize(x) for x in meta['column_names']]
  File "/usr/local/lib/python2.7/site-packages/quandl/util.py", line 26, in methodize
    return parameterize(string.replace(r'.', '')).replace(r'-', '_')
TypeError: an integer is required

The other examples (not involving "WIKI/*") seem to be working.

pandas dependency

pip install Quandl gives "ImportError: No module named pandas"

Downloading/unpacking Quandl
Downloading Quandl-2.2.tar.gz
Running setup.py (path:/home/hobs/.virtualenvs/pugtest/build/Quandl/setup.py) egg_info for package Quandl
Traceback (most recent call last):
File "", line 17, in
File "/home/hobs/.virtualenvs/pugtest/build/Quandl/setup.py", line 6, in
import Quandl
File "Quandl/init.py", line 13, in
from .Quandl import (
File "Quandl/Quandl.py", line 11, in
import pandas as pd
ImportError: No module named pandas
Complete output from command python setup.py egg_info:
Traceback (most recent call last):

File "", line 17, in

File "/home/hobs/.virtualenvs/pugtest/build/Quandl/setup.py", line 6, in

import Quandl

File "Quandl/init.py", line 13, in

from .Quandl import (

File "Quandl/Quandl.py", line 11, in

import pandas as pd

ImportError: No module named pandas

LICENSE file to be added

Can you please add the license file in the root of the repo? This is suggested to include the package on conda-forge.

Thank you.

Datatable filtering bug

Hi.

I think there might be a bug in the quandl.get_table when trying to filter the data. Specifically i try to filter the 'ZACKS/EEH' datase using the "obs_date" column to only download recent data. If i try to download the last 20 days the code below works fine, filters correctly, and i can see data up until yesterday. However trying to download for the e.g the last 10 days the code fails with the error:

QuandlError: (Status 500) Something went wrong. Please try again. If you continue to have problems, please contact us at [email protected].

Code example

database = {
    'dataset': 'ZACKS/EEH',
    'date_filter_column': 'obs_date'
}


import quandl
import datetime

n_days_back = 10 #change to e.g. 20 to make the code work

today = datetime.date.today()
start_delta = datetime.timedelta(days=n_days_back)
start_of_week = today - start_delta
start_date = "%d-%0.2d-%0.2d"%(start_of_week.year, start_of_week.month, start_of_week.day)
print(start_date)

options = {'datatable_code': database['dataset'],
           'paginate': True,
           database['date_filter_column']: {'gte': start_date}}
data = quandl.get_table(**options)

Traceback

QuandlError                               Traceback (most recent call last)
<ipython-input-149-cc398ea52228> in <module>()
     20            'paginate': True,
     21            database['date_filter_column']: {'gte': start_date}}
---> 22 data = quandl.get_table(**options)

/Users/casperkaae/anaconda3/lib/python3.6/site-packages/quandl/get_table.py in get_table(datatable_code, **options)
     17     while True:
     18         next_options = copy.deepcopy(options)
---> 19         next_data = Datatable(datatable_code).data(params=next_options)
     20 
     21         if data is None:

/Users/casperkaae/anaconda3/lib/python3.6/site-packages/quandl/model/datatable.py in data(self, **options)
     14     def data(self, **options):
     15         updated_options = Util.convert_options(**options)
---> 16         return Data.page(self, **updated_options)

/Users/casperkaae/anaconda3/lib/python3.6/site-packages/quandl/operations/list.py in page(cls, datatable, **options)
     22         params = {'id': str(datatable.code)}
     23         path = Util.constructed_path(datatable.default_path(), params)
---> 24         r = Connection.request('get', path, **options)
     25         response_data = r.json()
     26         Util.convert_to_dates(response_data)

/Users/casperkaae/anaconda3/lib/python3.6/site-packages/quandl/connection.py in request(cls, http_verb, url, **options)
     34         abs_url = '%s/%s' % (ApiConfig.api_base, url)
     35 
---> 36         return cls.execute_request(http_verb, abs_url, **options)
     37 
     38     @classmethod

/Users/casperkaae/anaconda3/lib/python3.6/site-packages/quandl/connection.py in execute_request(cls, http_verb, url, **options)
     42             response = func(url, **options)
     43             if response.status_code < 200 or response.status_code >= 300:
---> 44                 cls.handle_api_error(response)
     45             else:
     46                 return response

/Users/casperkaae/anaconda3/lib/python3.6/site-packages/quandl/connection.py in handle_api_error(cls, resp)
     64         # throw generic error
     65         if 'quandl_error' not in error_body:
---> 66             raise QuandlError(http_status=resp.status_code, http_body=resp.text)
     67 
     68         code = error_body['quandl_error']['code']

QuandlError: (Status 500) Something went wrong. Please try again. If you continue to have problems, please contact us at connect@quandl.com

Create Test Suite

This library could use some tests.

I'm happy to work on this (and have started on it, using unittest) but I'm wondering what the best way to go about doing this might be? I have a few questions:

Is there a particular dataset I should test against? To avoid flooding Quandl with test datasets (needed to test the "push" functionality), is there a way to programmatically delete datasets that are associated with an account/api key? Should there be a API key just for running tests? (Probably not, but otherwise how should we run them?)

Is anyone aware of any examples of good libraries wrapping around a service like Quandl (in whichever language, though Python would be ideal)? I'd love to learn.

3.01 required MS VC++ 1.0

pip wants to find MSVC++ - I have the complied version of Numpy installed and working (i'm trying to upgrade from 2.89)

requirements.txt does not allow for newer versions of numpy and pandas

Currently requirements.txt only allows for numpy 1.7.1 and pandas 0.11. Users with newer versions are therefore forced to install older versions of these modules. Requirements.txt should be changed from:
numpy==1.7.1
pandas==0.11

to:
numpy>=1.7.1
pandas>=0.11

This saves users from having to recompile an old version of numpy, if they already have a newer version installed.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.