kaggle / docker-python Goto Github PK

Kaggle Python docker image

License: Apache License 2.0

Python 81.30% Shell 9.50% Smarty 0.21% Dockerfile 6.00% Jupyter Notebook 0.83% PureBasic 2.16%

docker-python's Introduction

docker-python

Kaggle Notebooks allow users to run a Python Notebook in the cloud against our competitions and datasets without having to download data or set up their environment.

This repository includes the Dockerfile for building the CPU-only and GPU image that runs Python Notebooks on Kaggle.

Our Python Docker images are stored on the Google Container Registry at:

CPU-only: gcr.io/kaggle-images/python
GPU: gcr.io/kaggle-gpu-images/python

Requesting new packages

First, evaluate whether installing the package yourself in your own notebooks suits your needs. See guide.

If you the first step above doesn't work for your use case, open an issue or a pull request.

Opening a pull request

Edit the Dockerfile.
Follow the instructions below to build a new image.
Add tests for your new package. See this example.
Follow the instructions below to test the new image.
Open a PR on this repo and you are all set!

Building a new image

./build

Flags:

--gpu to build an image for GPU.
--use-cache for faster iterative builds.

Testing a new image

A suite of tests can be found under the /tests folder. You can run the test using this command:

./test

Flags:

--gpu to test the GPU image.
--pattern test_keras.py or -p test_keras.py to run a single test
--image gcr.io/kaggle-images/python:ci-pretest or -i gcr.io/kaggle-images/python:ci-pretest to test against a specific image

Running the image

For the CPU-only image:

# Run the image built locally:
docker run --rm -it kaggle/python-build /bin/bash
# Run the pre-built image from gcr.io
docker run --rm -it gcr.io/kaggle-images/python /bin/bash

For the GPU image:

# Run the image built locally:
docker run --runtime nvidia --rm -it kaggle/python-gpu-build /bin/bash
# Run the image pre-built image from gcr.io
docker run --runtime nvidia --rm -it gcr.io/kaggle-gpu-images/python /bin/bash

To ensure your container can access the GPU, follow the instructions posted here.

docker-python's People

Contributors

Stargazers

Watchers

Forkers

luminescent nkhuyu anlthms larryhillyer satpreetsingh solesschong tokestermw rezhajulio asyapronina copyfun h2oai gprasa7s urandon mr1azl alexandercbooth ferrarisf50 caohy1988 maveriq yifanxie atenni suryams gbmeow wangzijia kiransom michelealberto thefon sambozek wangz10 mlopezm joseamunozmata hrithik96 lindauruchurtu yassinealouini rtarun larsaurdal kentchun33333 rubik franklinma810 scottsunsh scollins83 k-kawa gallileog astrovale kretes wujiahongpku gidler twisniewski samucc wjpjet amanbh galtay ljx0520 mithrillion gth158a jaime-cn dominicbreuker chschroeder mrisdal kmader rodrigogonzalez gp7029 jdduryea tvanesse drawar flaviostutz mboker bityangke drpelky ryokamiya thomzi12 evrial pycn kvasagiri enterstudio enterstudios paulhendricks mqwutom mpower4ru mayatp bornfreesoul whatevergeek mengmengmiao phuongpopo chaos66ren nathanie climbsrocks huaxiatiankong robinkraft jeperez woodhaha njwilson23 deejmore qrks manavsehgal sachdeva-shubham mariosilvaecastro awtotty ryanjameslett bionicles busrafenerci

docker-python's Issues

Add new package - sklearn-pandas

Hello Kaggle team,

Can you add the sklearn-pandas package.
Command is "pip install sklearn-pandas" but I'm unable to test it in docker for the moment

Sklearn-pandas provides a bridge between Scikit-Learn’s machine learning methods and pandas-style Data Frames.
Doc: https://github.com/paulgb/sklearn-pandas

Thanks

how to use gpu with theano or tensorflow

theano is using cpu right now

PySpark support for Kaggle.

Can Kaggle docker support PySpark? PySpark can help to do transformation easily.

Dask-ml

Dask-ML enables parallel and distributed machine learning using Dask alongside existing machine learning libraries like Scikit-Learn.

Src: https://github.com/dask/dask-ml
Docs: https://dask-ml.readthedocs.io/en/latest/

...

XGBoost, TensorFlow, scikit-learn

how to use GPU

I pulled kaggle python image and seems to work fine, but I'm not sure what to do to use my GPU with theano or tensorflow.

Is it as simple as changing flags in .theanorc file? Can i edit it directly in the docker image?

New to docker here :)

Not able to import some of the packages listed in Dockerfile

I noticed that I'm not able to import packages that are added recently in the Dockerfile. Basically, I've tried the following packages and failed to import any of them:

pip install pandas-datareader && \ pip install pykoko && \ pip install featuretools && \ pip install wordsegment && \ pip install pyahocorasick && \ pip install wordbatch && \ pip install emoji && \

Does this have to do with how frequently the changes in the Dockerfile is reflected on Kaggle kernels?

Add new package - mlxtend

Hello Kaggle team,

Can you add the mlxtend package.
Command is "pip install mlxtend" but I'm unable to test it in docker for the moment

Mlxtend extends Scikit-learn with an easy to use interface for Ensembling, Voting and Stacking

Doc: https://rasbt.github.io/mlxtend/

Thanks

Add support for pip in online Kaggle kernels

It would be nice to have ability to use pip to install new packages here online on Kaggle kernel

h2o import fails

Hi guys,

I have just pulled recent image of kaggle/python from docker repo, and I face the following problem when trying to import h2o:

In [1]: import h2o
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-707498249ab8> in <module>()
----> 1 import h2o

/opt/conda/lib/python3.4/site-packages/h2o/__init__.py in <module>()
    398 __version__ = "3.6.0.8"
    399 from h2o import *
--> 400 from model import *
    401 from demo import *
    402 from h2o_logging import *

ImportError: No module named 'model'

I have checked and same issue is on kaggle scripts, when running it on kaggle website

GDAL library for processing map data...

I'm looking at OpenStreetMap data for the Sberbank competition and saw this article http://andrewgaidus.com/Convert_OSM_Data/ that led me to GDAL (which has 'ogr' to read geodata files including OSM format) - https://pypi.python.org/pypi/GDAL

edit: It does seem to require 'gdal-bin' and 'libgdal-dev' in ubuntu, then running pip with "CPLUS_INCLUDE_PATH=/usr/include/gdal C_INCLUDE_PATH=/usr/include/gdal" environment variabls so it might pull in too much to be practical...

Add wfdb package

(Copying text from PR #162)

There are many datasets in the physionet physiobank database with physiologic signals and related data for use by the biomedical research community.

Some data has already been uploaded to kaggle data sets but it's hard to use because it is not in the right format:
https://www.kaggle.com/ecerulm/apneaecg/discussion/33856

By adding the wfdb package to the kernels it will allow users to explore this data set and more. There are lots of really interesting machine learning challenges in these data sets.

Add some packages

Hi,

Can you add

mlxtend : https://github.com/rasbt/mlxtend

and

scikit-neuralnetwork: https://scikit-neuralnetwork.readthedocs.io/en/latest/

Please ?

In otherhand, I tried this to do it myself but it didn' suceed:

docker run -it kaggle/python /bin/bash
and in the bash : pip install mlxtend but import mlxtend still doesn't work. What I am missing ?

Regards

How some kernels run more than 3600 seconds

Sometimes it runs more than an hour. That extra runtime gives me false hopes that next time also it will run the whole script but after waiting for 1 hour it fails saying it exceeded 3600s.

Can someone explain this erratic behaviour?

Kernel ran 9800secs: https://www.kaggle.com/mkagenius/adcom05-celiktutan-cellphone-pdf-2-feats?scriptVersionId=2045804

Is it possible that if you run a notebook, each cell gets 3600 or more if you are active on the page?

Add Entry Point / CMD for Dockerfile?

In order to run the Dockerfile with Binder it needs to have an ENTRYPOINT or CMD at the end, is it possible to add this?

PyTorch fails on import

Not sure if this is the place to report it, but it seems like trying to import pytorch fails completely atm in kernels. Even if it's the first and only import.

 [1]:

import torch
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-c031d3dd82fc> in <module>()
----> 1 import torch

/opt/conda/lib/python3.6/site-packages/torch/__init__.py in <module>()
     51 sys.setdlopenflags(_dl_flags.RTLD_GLOBAL | _dl_flags.RTLD_NOW)
     52 
---> 53 from torch._C import *
     54 
     55 __all__ += [name for name in dir(_C)

ImportError: dlopen: cannot load any more object with static TLS

Please install numpy first

Running current environment, pyfasttext raise below error.
Traceback (most recent call last): File "__init__.pxd", line 987, in numpy.import_array (src/pyfasttext.cpp:15915) RuntimeError: module compiled against API version 0xb but this version of numpy is 0xa

In dockerfile, new contributions are written line 234-370.
https://github.com/Kaggle/docker-python/blob/master/Dockerfile#L234-L378

However, numpy (which is basic array library for python), is called at line 383.
https://github.com/Kaggle/docker-python/blob/master/Dockerfile#L383.

In my opiniton, the cause of this errors is installing order. Generary speaking, basic library should be installed first. I think many type of libraries which have depandancy with numpy have troubles now.

Since this problem is outside of contribution area (line 234-370), I want Kaggle Team to handle it.

Add nibabel for medical image formats like Nifti

As a pure-python package nibabel (http://nipy.org/nibabel) should not cause many conflicts and would make opening medical image files much easier (nifti is smaller and more standardized for 3D images than dicom)

Can not found "input" folder

Hello,

executing a kernel i got an error FileNotFoundError. It happens because of the container does not have input folder with kaggle's datasets.

Add support for conda in online Kaggle kernels

It would be nice to use ability to install new packages via conda package manager (https://conda.io/docs/intro.html) in online Kaggle kernels.

qtconsole in notebook

I find that %qtconsole isn't working for me. The little python console is a very handy way to look at the data and experiment with it as I build and test my notebooks.

SimpleITK?

For the 2017 Kaggle Data Science Bowl the tutorial (https://github.com/booz-allen-hamilton/DSB3Tutorial) uses the SimpleITK package for loading and processing DICOMS a tool that is much better suited than pydicom for 3D data. Can this be added to the Docker Image / Kernels?

skimage cannot be imported

When running
from skimage.util.montage import montage2d the following error gets thrown in the log file

Intel MKL FATAL ERROR: Cannot load libmkl_avx2.so or libmkl_def.so.

Would like to have psycopg2 (Postgres connector)...

I'm playing with SQL stuff for a competition and am manually loading that in now. Installs with a clean "pip install psycopg2" - thanks!

Add some packages: md5

is it possible for you to add the md5 package to be able to use this please ?

https://github.com/msegala/Ensemble-Tools/blob/master/blend_proba.py

Thanks

matplotlib imshow function does not work

I am using kaggle/python image in docker to run a kaggle kernel but imshow does not work on my system. nothing is shown, even no plot window pops up. I have the ubuntu 16, pycharm 2017 connected to the latest docker version. any suggestion?

Add pretrained model weights for Keras

Keras has a keras.applications module with pretrained models and automatically downloads their weights. Since the Kaggle Kernels have no external internet access, these cannot be downloaded. It would however work to add these weights to the docker-image so they were always available (since keras first checks a local cache before downloading, https://github.com/fchollet/keras/blob/master/keras/applications/vgg16.py#L162) https://github.com/fchollet/keras/blob/master/keras/utils/data_utils.py#L117). Particularly as a number of the tasks from the YouTube challenge to the Cervical Cancer contest would benefit from using pretrained models to create feature vectors

dash plotly

now that we are having a data visualization / kernel competition it wld be nice to enable plotly dash (https://plot.ly/products/dash/) on kernels

Add lightgbm python bindings

Hi,
please add the lightgbm python package
https://github.com/Microsoft/LightGBM/tree/master/examples/python-guide

Reason:
3-8 times faster than xgboost, potentially more accurate and less memory required
https://github.com/Microsoft/LightGBM/wiki/Experiments#comparison-experiment
Tnx

ImportError when running jupyter notebook

Hi,

I get an ImportError when trying to start a jupyter notebook on a fresh pull of kaggle/python:

$ docker run --rm -it kaggle/python jupyter notebook
Traceback (most recent call last):
  File "/opt/conda/bin/jupyter-notebook", line 4, in <module>
    import notebook.notebookapp
ImportError: No module named 'notebook'

Entering the container and running conda install jupyter fixes the issue:

root@2b56afbe1057:/# conda install jupyter
Fetching package metadata: ....
Solving package specifications: .........

Package plan for installation in environment /opt/conda:

The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    jsonschema-2.4.0           |           py34_0          53 KB
    mistune-0.7.1              |           py34_0         649 KB
    jupyter_core-4.0.6         |           py34_0          49 KB
    terminado-0.5              |           py34_1          18 KB
    jupyter_client-4.1.1       |           py34_0          91 KB
    nbformat-4.0.1             |           py34_0         114 KB
    ipykernel-4.2.2            |           py34_0         116 KB
    nbconvert-4.1.0            |           py34_0         275 KB
    jupyter_console-4.1.0      |           py34_0          25 KB
    notebook-4.1.0             |           py34_0         4.4 MB
    qtconsole-4.1.1            |           py34_0         129 KB
    ipywidgets-4.1.1           |           py34_0         100 KB
    jupyter-1.0.0              |           py34_2           2 KB
    ------------------------------------------------------------
                                           Total:         5.9 MB

The following packages will be UPDATED:

    ipykernel:       4.2.2-py35_0 --> 4.2.2-py34_0
    ipywidgets:      4.1.1-py35_0 --> 4.1.1-py34_0
    jsonschema:      2.4.0-py35_0 --> 2.4.0-py34_0
    jupyter:         1.0.0-py35_1 --> 1.0.0-py34_2
    jupyter_client:  4.1.1-py35_0 --> 4.1.1-py34_0
    jupyter_console: 4.1.0-py35_0 --> 4.1.0-py34_0
    jupyter_core:    4.0.6-py35_0 --> 4.0.6-py34_0
    mistune:         0.7.1-py35_0 --> 0.7.1-py34_0
    nbconvert:       4.1.0-py35_0 --> 4.1.0-py34_0
    nbformat:        4.0.1-py35_0 --> 4.0.1-py34_0
    notebook:        4.1.0-py35_0 --> 4.1.0-py34_0
    qtconsole:       4.1.1-py35_0 --> 4.1.1-py34_0
    terminado:       0.5-py35_1   --> 0.5-py34_1

Could this be related to having Python 3.5 packages installed when the container currently runs Python 3.4? (Eg. see notebook on the third last line above.)

Thanks,
Andrew.

upgrade hypertools to 0.3.0

Hi - we just added a bunch of new stuff to our software (no API changes). Could you pull the latest version of hypertools from pip? Thank you!

module Geohash do not work

The module does not work whit python3 (issue -> vinsci/geohash#4).

I also tried to install from code source and i have the same error.

However the following fix works :
sed -i -- 's/geohash/.geohash/g' /opt/conda/lib/python3.6/site-packages/Geohash/init.py

Unable to pull this image: "No space left on device"

Hi there.

I tried pulling this image this evening and received this error about halfway through:

failed to register layer: ApplyLayer exit status 1 stdout:  stderr: write /usr/local/src/opencv/build/modules/video/perf_precomp.hpp.gch/opencv_perf_video_RELEASE.gch: no space left on device

What's interesting is, I know that I am not out of space. I'm running arch linux as my host, and I just finished creating a 1TB partition exclusively for the /var directory on my machine. Before the pull there was 869.9 gigs of space available.

Any suggestions on what I should do here?

Thanks. I appreciate the help!

IPython.lib.display.IFrame not working

When running code like the following in a notebook :

from IPython.display import IFrame
IFrame('http://stackoverflow.org', width=700, height=350)

There is no output.

Is this behaviour intended??

Failed to build vowpalwabbit

Collecting vowpalwabbit

Downloading vowpalwabbit-8.4.0.tar.gz (10.2MB)

Building wheels for collected packages: vowpalwabbit

Running setup.py bdist_wheel for vowpalwabbit: started

Running setup.py bdist_wheel for vowpalwabbit: still running...

Running setup.py bdist_wheel for vowpalwabbit: finished with status 'error'

Complete output from command /opt/conda/bin/python -u -c "import setuptools, tokenize;file='/tmp/pip-build-vs7jgzog/vowpalwabbit/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" bdist_wheel -d /tmp/tmp7jaam29ppip-wheel- --python-tag cp36:

Failed building wheel for vowpalwabbit

running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/vowpalwabbit
copying vowpalwabbit/pyvw.py -> build/lib.linux-x86_64-3.6/vowpalwabbit
copying vowpalwabbit/init.py -> build/lib.linux-x86_64-3.6/vowpalwabbit
copying vowpalwabbit/sklearn_vw.py -> build/lib.linux-x86_64-3.6/vowpalwabbit
running egg_info
writing vowpalwabbit.egg-info/PKG-INFO
writing dependency_links to vowpalwabbit.egg-info/dependency_links.txt
writing top-level names to vowpalwabbit.egg-info/top_level.txt
warning: manifest_maker: standard file '-c' not found

reading manifest file 'vowpalwabbit.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '.dll' under directory 'bin'
warning: no previously-included files matching '.o' found anywhere in distribution
warning: no previously-included files matching '.exe' found anywhere in distribution
warning: no previously-included files matching '.pyc' found anywhere in distribution
writing manifest file 'vowpalwabbit.egg-info/SOURCES.txt'
running build_ext
cd vowpalwabbit; make -j 8 things
make[1]: Entering directory '/tmp/pip-build-vs7jgzog/vowpalwabbit/src/vowpalwabbit'
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c main.cc -o main.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c hash.cc -o hash.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c global_data.cc -o global_data.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c io_buf.cc -o io_buf.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c parse_regressor.cc -o parse_regressor.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c parse_primitives.cc -o parse_primitives.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c unique_sort.cc -o unique_sort.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c cache.cc -o cache.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c rand48.cc -o rand48.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c simple_label.cc -o simple_label.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c multiclass.cc -o multiclass.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c oaa.cc -o oaa.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c multilabel_oaa.cc -o multilabel_oaa.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c boosting.cc -o boosting.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c ect.cc -o ect.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c marginal.cc -o marginal.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c autolink.cc -o autolink.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c binary.cc -o binary.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c lrq.cc -o lrq.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c cost_sensitive.cc -o cost_sensitive.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c multilabel.cc -o multilabel.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c label_dictionary.cc -o label_dictionary.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c csoaa.cc -o csoaa.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c cb.cc -o cb.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c cb_adf.cc -o cb_adf.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c cb_algs.cc -o cb_algs.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c search.cc -o search.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c search_meta.cc -o search_meta.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c search_sequencetask.cc -o search_sequencetask.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c search_dep_parser.cc -o search_dep_parser.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c search_hooktask.cc -o search_hooktask.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c search_multiclasstask.cc -o search_multiclasstask.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c search_entityrelationtask.cc -o search_entityrelationtask.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c search_graph.cc -o search_graph.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c parse_example.cc -o parse_example.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c scorer.cc -o scorer.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c network.cc -o network.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c parse_args.cc -o parse_args.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c accumulate.cc -o accumulate.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c gd.cc -o gd.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c learner.cc -o learner.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c mwt.cc -o mwt.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c lda_core.cc -o lda_core.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c gd_mf.cc -o gd_mf.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c mf.cc -o mf.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c bfgs.cc -o bfgs.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c noop.cc -o noop.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c print.cc -o print.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c example.cc -o example.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c parser.cc -o parser.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c loss_functions.cc -o loss_functions.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c sender.cc -o sender.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c nn.cc -o nn.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c confidence.cc -o confidence.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c bs.cc -o bs.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c cbify.cc -o cbify.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c explore_eval.cc -o explore_eval.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c topk.cc -o topk.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c stagewise_poly.cc -o stagewise_poly.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c log_multi.cc -o log_multi.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c recall_tree.cc -o recall_tree.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c active.cc -o active.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c active_cover.cc -o active_cover.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c kernel_svm.cc -o kernel_svm.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c best_constant.cc -o best_constant.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c ftrl.cc -o ftrl.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c svrg.cc -o svrg.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c lrqfa.cc -o lrqfa.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c interact.cc -o interact.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c comp_io.cc -o comp_io.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c interactions.cc -o interactions.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c vw_exception.cc -o vw_exception.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c vw_validate.cc -o vw_validate.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c audit_regressor.cc -o audit_regressor.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c gen_cs_example.cc -o gen_cs_example.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c cb_explore.cc -o cb_explore.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c action_score.cc -o action_score.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c cb_explore_adf.cc -o cb_explore_adf.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c OjaNewton.cc -o OjaNewton.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c parse_example_json.cc -o parse_example_json.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c baseline.cc -o baseline.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c classweight.cc -o classweight.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c allreduce_sockets.cc -o allreduce_sockets.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -c allreduce_threads.cc -o allreduce_threads.o
ar rcs liballreduce.a allreduce_sockets.o allreduce_threads.o vw_exception.o
ar rcs libvw.a hash.o global_data.o io_buf.o parse_regressor.o parse_primitives.o unique_sort.o cache.o rand48.o simple_label.o multiclass.o oaa.o multilabel_oaa.o boosting.o ect.o marginal.o autolink.o binary.o lrq.o cost_sensitive.o multilabel.o label_dictionary.o csoaa.o cb.o cb_adf.o cb_algs.o search.o search_meta.o search_sequencetask.o search_dep_parser.o search_hooktask.o search_multiclasstask.o search_entityrelationtask.o search_graph.o parse_example.o scorer.o network.o parse_args.o accumulate.o gd.o learner.o mwt.o lda_core.o gd_mf.o mf.o bfgs.o noop.o print.o example.o parser.o loss_functions.o sender.o nn.o confidence.o bs.o cbify.o explore_eval.o topk.o stagewise_poly.o log_multi.o recall_tree.o active.o active_cover.o kernel_svm.o best_constant.o ftrl.o svrg.o lrqfa.o interact.o comp_io.o interactions.o vw_exception.o vw_validate.o audit_regressor.o gen_cs_example.o cb_explore.o action_score.o cb_explore_adf.o OjaNewton.o parse_example_json.o baseline.o classweight.o
/usr/bin/g++ -std=c++0x -D__extern_always_inline=inline -Wall -pedantic -DNDEBUG -O3 -fomit-frame-pointer -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -I /usr/include -I ../rapidjson/include -fPIC -I ../rapidjson/include -o vw main.o -L. -l vw -l allreduce -L /usr/local/lib -L /usr/lib -L /usr/lib/x86_64-linux-gnu -l boost_program_options -l pthread -l z
make[1]: Leaving directory '/tmp/pip-build-vs7jgzog/vowpalwabbit/src/vowpalwabbit'
cd python; make things
make[1]: Entering directory '/tmp/pip-build-vs7jgzog/vowpalwabbit/src/python'
Using Python 3.6
/usr/bin/g++ -std=c++0x -I/opt/conda/include/python3.6m -I/opt/conda/include/python3.6m -I /usr/include -I ../rapidjson/include -fPIC -c pylibvw.cc -o pylibvw.o
/usr/bin/g++ -shared -Wl,--export-dynamic pylibvw.o -L /usr/local/lib -L /usr/lib -L /usr/lib/x86_64-linux-gnu -L/opt/conda/lib/python3.6/config-3.6m-x86_64-linux-gnu -L/opt/conda/lib -lpython3.6m -lpthread -ldl -lutil -lrt -lm -Xlinker -export-dynamic ../vowpalwabbit/libvw.a ../vowpalwabbit/liballreduce.a -L /usr/local/lib -L /usr/lib -L /usr/lib/x86_64-linux-gnu -l boost_program_options -l pthread -l z -l boost_python-py34 -o pylibvw.so
lto1: fatal error: bytecode stream generated with LTO version 6.0 instead of the expected 3.0
compilation terminated.
lto-wrapper: /usr/bin/g++ returned 1 exit status
/usr/bin/ld: lto-wrapper failed
collect2: error: ld returned 1 exit status
Makefile:55: recipe for target 'pylibvw.so' failed
make[1]: *** [pylibvw.so] Error 1
make[1]: Leaving directory '/tmp/pip-build-vs7jgzog/vowpalwabbit/src/python'
Makefile:118: recipe for target 'python' failed
make: *** [python] Error 2
Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-build-vs7jgzog/vowpalwabbit/setup.py", line 211, in
tests_require=['tox']
File "/opt/conda/lib/python3.6/distutils/core.py", line 148, in setup
dist.run_commands()
File "/opt/conda/lib/python3.6/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/opt/conda/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/opt/conda/lib/python3.6/site-packages/wheel/bdist_wheel.py", line 179, in run
self.run_command('build')
File "/opt/conda/lib/python3.6/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/opt/conda/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/opt/conda/lib/python3.6/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/opt/conda/lib/python3.6/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/opt/conda/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/opt/conda/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/setuptools/command/build_ext.py", line 77, in run
File "/opt/conda/lib/python3.6/site-packages/Cython/Distutils/old_build_ext.py", line 185, in run
_build_ext.build_ext.run(self)
File "/opt/conda/lib/python3.6/distutils/command/build_ext.py", line 339, in run
self.build_extensions()
File "/opt/conda/lib/python3.6/site-packages/Cython/Distutils/old_build_ext.py", line 193, in build_extensions
self.build_extension(ext)
File "/tmp/pip-build-vs7jgzog/vowpalwabbit/setup.py", line 122, in build_extension
subprocess.check_call(['make', 'python'], cwd=path.join(here, 'src'), env=env)
File "/opt/conda/lib/python3.6/subprocess.py", line 291, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['make', 'python']' returned non-zero exit status 2.

Running setup.py clean for vowpalwabbit

Failed to build vowpalwabbit

Installing collected packages: vowpalwabbit

Running setup.py install for vowpalwabbit: started

Running setup.py install for vowpalwabbit: finished with status 'error'
Complete output from command /opt/conda/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-vs7jgzog/vowpalwabbit/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-aaw1oa16-record/install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/vowpalwabbit
copying vowpalwabbit/pyvw.py -> build/lib.linux-x86_64-3.6/vowpalwabbit
copying vowpalwabbit/__init__.py -> build/lib.linux-x86_64-3.6/vowpalwabbit
copying vowpalwabbit/sklearn_vw.py -> build/lib.linux-x86_64-3.6/vowpalwabbit
running egg_info
creating vowpalwabbit.egg-info
writing vowpalwabbit.egg-info/PKG-INFO
writing dependency_links to vowpalwabbit.egg-info/dependency_links.txt
writing top-level names to vowpalwabbit.egg-info/top_level.txt
writing manifest file 'vowpalwabbit.egg-info/SOURCES.txt'
warning: manifest_maker: standard file '-c' not found

reading manifest file 'vowpalwabbit.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '*' under directory 'src'
warning: no files found matching '*.dll' under directory 'bin'
warning: no previously-included files matching '*.o' found anywhere in distribution
warning: no previously-included files matching '*.exe' found anywhere in distribution
warning: no previously-included files matching '*.pyc' found anywhere in distribution
writing manifest file 'vowpalwabbit.egg-info/SOURCES.txt'
running build_ext
make: *** No rule to make target 'clean'.  Stop.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/pip-build-vs7jgzog/vowpalwabbit/setup.py", line 211, in <module>
    tests_require=['tox']
  File "/opt/conda/lib/python3.6/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/opt/conda/lib/python3.6/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/opt/conda/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/opt/conda/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/setuptools/command/install.py", line 61, in run
  File "/opt/conda/lib/python3.6/distutils/command/install.py", line 545, in run
    self.run_command('build')
  File "/opt/conda/lib/python3.6/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/opt/conda/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/opt/conda/lib/python3.6/distutils/command/build.py", line 135, in run
    self.run_command(cmd_name)
  File "/opt/conda/lib/python3.6/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/opt/conda/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/opt/conda/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/setuptools/command/build_ext.py", line 77, in run
  File "/opt/conda/lib/python3.6/site-packages/Cython/Distutils/old_build_ext.py", line 185, in run
    _build_ext.build_ext.run(self)
  File "/opt/conda/lib/python3.6/distutils/command/build_ext.py", line 339, in run
    self.build_extensions()
  File "/opt/conda/lib/python3.6/site-packages/Cython/Distutils/old_build_ext.py", line 193, in build_extensions
    self.build_extension(ext)
  File "/tmp/pip-build-vs7jgzog/vowpalwabbit/setup.py", line 105, in build_extension
    prep()
  File "/tmp/pip-build-vs7jgzog/vowpalwabbit/setup.py", line 65, in prep
    subprocess.check_call(['make', 'clean'], cwd=path.join(here, 'src', 'python'))
  File "/opt/conda/lib/python3.6/subprocess.py", line 291, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['make', 'clean']' returned non-zero exit status 2.

----------------------------------------

Command "/opt/conda/bin/python -u -c "import setuptools, tokenize;file='/tmp/pip-build-vs7jgzog/vowpalwabbit/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-aaw1oa16-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-vs7jgzog/vowpalwabbit/

Error: ResponseItem.ErrorDetail[code=1,message=The command '/bin/sh -c sed -i "s/httpredir.debian.org/debian.uchicago.edu/" /etc/apt/sources.list && apt-get update && apt-get install -y build-essential && conda update -y conda && conda update -y python && pip install tensorflow && apt-get install -y libboost-program-options-dev zlib1g-dev libboost-python-dev && cd /usr/lib/x86_64-linux-gnu/ && rm -f libboost_python.a && rm -f libboost_python.so && ln -sf libboost_python-py34.so libboost_python.so && ln -sf libboost_python-py34.a libboost_python.a && pip install vowpalwabbit && pip install seaborn python-dateutil dask pytagcloud pyyaml ggplot joblib husl geopy ml_metrics mne pyshp gensim && conda install -y -c conda-forge spacy && python -m spacy download en && apt-get -y install dbus fontconfig fontconfig-config fonts-dejavu-core fonts-droid ghostscript gsfonts hicolor-icon-theme libavahi-client3 libavahi-common-data libavahi-common3 libcairo2 libcap-ng0 libcroco3 libcups2 libcupsfilters1 libcupsimage2 libdatrie1 libdbus-1-3 libdjvulibre-text libdjvulibre21 libfftw3-double3 libfontconfig1 libfreetype6 libgdk-pixbuf2.0-0 libgdk-pixbuf2.0-common libgomp1 libgraphite2-3 libgs9 libgs9-common libharfbuzz0b libijs-0.35 libilmbase6 libjasper1 libjbig0 libjbig2dec0 libjpeg62-turbo liblcms2-2 liblqr-1-0 libltdl7 libmagickcore-6.q16-2 libmagickcore-6.q16-2-extra libmagickwand-6.q16-2 libnetpbm10 libopenexr6 libpango-1.0-0 libpangocairo-1.0-0 libpangoft2-1.0-0 libpaper-utils libpaper1 libpixman-1-0 libpng12-0 librsvg2-2 librsvg2-common libthai-data libthai0 libtiff5 libwmf0.2-7 libxcb-render0 libxcb-shm0 netpbm poppler-data p7zip-full && cd /usr/local/src && wget http://transloadit.imagemagick.org/download/ImageMagick.tar.gz && tar xzf ImageMagick.tar.gz && cd ls -d ImageMagick-* && pwd && ls -al && ./configure && make -j $(nproc) && make install && cd ../ && rm -rf ImageMagick* && apt-get -y install libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev && apt-get -y install libtbb2 libtbb-dev libjpeg-dev libtiff-dev libjasper-dev && apt-get -y install cmake && cd /usr/local/src && git clone --depth 1 https://github.com/Itseez/opencv.git && cd opencv && mkdir build && cd build && cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D WITH_TBB=ON -D WITH_FFMPEG=OFF -D WITH_V4L=ON -D WITH_QT=OFF -D WITH_OPENGL=ON -D PYTHON3_LIBRARY=/opt/conda/lib/libpython3.6m.so -D PYTHON3_INCLUDE_DIR=/opt/conda/include/python3.6m/ -D PYTHON_LIBRARY=/opt/conda/lib/libpython3.6m.so -D PYTHON_INCLUDE_DIR=/opt/conda/include/python3.6m/ -D BUILD_PNG=TRUE .. && make -j $(nproc) && make install && echo "/usr/local/lib/python3.6/site-packages" > /etc/ld.so.conf.d/opencv.conf && ldconfig && cp /usr/local/lib/python3.6/site-packages/cv2.cpython-36m-x86_64-linux-gnu.so /opt/conda/lib/python3.6/site-packages/ && rm -rf /usr/local/src/opencv && rm -rf /root/.cache/pip/* && apt-get autoremove -y && apt-get clean' returned a non-zero code: 1]
Failed to deploy ' Dockerfile: Dockerfile': The command '/bin/sh -c sed -i "s/httpredir.debian.org/debian.uchicago.edu/" /etc/apt/sources.list && apt-get update && apt-get install -y build-essential && conda update -y conda && conda update -y python && pip install tensorflow && apt-get install -y libboost-program-options-dev zlib1g-dev libboost-python-dev && cd /usr/lib/x86_64-linux-gnu/ && rm -f libboost_python.a && rm -f libboost_python.so && ln -sf libboost_python-py34.so libboost_python.so && ln -sf libboost_python-py34.a libboost_python.a && pip install vowpalwabbit && pip install seaborn python-dateutil dask pytagcloud pyyaml ggplot joblib husl geopy ml_metrics mne pyshp gensim && conda install -y -c conda-forge spacy && python -m spacy download en && apt-get -y install dbus fontconfig fontconfig-config fonts-dejavu-core fonts-droid ghostscript gsfonts hicolor-icon-theme libavahi-client3 libavahi-common-data libavahi-common3 libcairo2 libcap-ng0 libcroco3 libcups2 libcupsfilters1 libcupsimage2 libdatrie1 libdbus-1-3 libdjvulibre-text libdjvulibre21 libfftw3-double3 libfontconfig1 libfreetype6 libgdk-pixbuf2.0-0 libgdk-pixbuf2.0-common libgomp1 libgraphite2-3 libgs9 libgs9-common libharfbuzz0b libijs-0.35 libilmbase6 libjasper1 libjbig0 libjbig2dec0 libjpeg62-turbo liblcms2-2 liblqr-1-0 libltdl7 libmagickcore-6.q16-2 libmagickcore-6.q16-2-extra libmagickwand-6.q16-2 libnetpbm10 libopenexr6 libpango-1.0-0 libpangocairo-1.0-0 libpangoft2-1.0-0 libpaper-utils libpaper1 libpixman-1-0 libpng12-0 librsvg2-2 librsvg2-common libthai-data libthai0 libtiff5 libwmf0.2-7 libxcb-render0 libxcb-shm0 netpbm poppler-data p7zip-full && cd /usr/local/src && wget http://transloadit.imagemagick.org/download/ImageMagick.tar.gz && tar xzf ImageMagick.tar.gz && cd ls -d ImageMagick-* && pwd && ls -al && ./configure && make -j $(nproc) && make install && cd ../ && rm -rf ImageMagick* && apt-get -y install libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev && apt-get -y install libtbb2 libtbb-dev libjpeg-dev libtiff-dev libjasper-dev && apt-get -y install cmake && cd /usr/local/src && git clone --depth 1 https://github.com/Itseez/opencv.git && cd opencv && mkdir build && cd build && cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D WITH_TBB=ON -D WITH_FFMPEG=OFF -D WITH_V4L=ON -D WITH_QT=OFF -D WITH_OPENGL=ON -D PYTHON3_LIBRARY=/opt/conda/lib/libpython3.6m.so -D PYTHON3_INCLUDE_DIR=/opt/conda/include/python3.6m/ -D PYTHON_LIBRARY=/opt/conda/lib/libpython3.6m.so -D PYTHON_INCLUDE_DIR=/opt/conda/include/python3.6m/ -D BUILD_PNG=TRUE .. && make -j $(nproc) && make install && echo "/usr/local/lib/python3.6/site-packages" > /etc/ld.so.conf.d/opencv.conf && ldconfig && cp /usr/local/lib/python3.6/site-packages/cv2.cpython-36m-x86_64-linux-gnu.so /opt/conda/lib/python3.6/site-packages/ && rm -rf /usr/local/src/opencv && rm -rf /root/.cache/pip/* && apt-get autoremove -y && apt-get clean' returned a non-zero code: 1

Plotly is not availalbe

When I pull the container from dockerhub or build it with this script the plotly package isn't available. I have to manually install it by running and committing to the container. I can see where the pip install plotly is happening but I'm not sure why it isn't getting installed into the container on dockerhub.

Thanks!

Please install Rtree

It would be great to have the "Rtree" package available for kaggle python kernels. It implements spatial indexing, which can speed up a lot of types of geospatial processing by a couple of orders of magnitude. Thanks!

allow bokeh js to load from cdns

Hi,
At the moment using the bokeh package in kaggle notebooks won't actually render the plots as the page/firewall/webserver blocks loading the js from bokeh's default cdn servers. This comes in the way of interactive(zoom, pan) visualizations that bokeh can provide.
So you can either load bokeh js, from that cdn, or host it in kaggle's cdn and redirect requests to that cdn to kaggle's .

Python 2.7 support

Some packages are written for Python 2 and throw errors when ran in python 3. Are you planning to support both versions of python or just stick to 3.5? Thanks.

Vowpal Wabbit failing import

import vowpalwabbit # nothing happens

from vowpalwabbit import pyvw  #fails
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-10-76d9c58cae1b> in <module>()
----> 1 from vowpalwabbit import pyvw

/opt/conda/lib/python3.6/site-packages/vowpalwabbit/pyvw.py in <module>()
      1 from __future__ import division
----> 2 import pylibvw
      3 
      4 
      5 class SearchTask():

ImportError: dynamic module does not define module export function (PyInit_pylibvw)

from vowpalwabbit.sklearn_vw import VWClassifier # fails
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-11-14a6a5c77f64> in <module>()
----> 1 from vowpalwabbit.sklearn_vw import VWClassifier

/opt/conda/lib/python3.6/site-packages/vowpalwabbit/sklearn_vw.py in <module>()
     16 from sklearn.utils.validation import check_is_fitted
     17 from sklearn.externals import joblib
---> 18 from vowpalwabbit import pyvw
     19 
     20 

/opt/conda/lib/python3.6/site-packages/vowpalwabbit/pyvw.py in <module>()
      1 from __future__ import division
----> 2 import pylibvw
      3 
      4 
      5 class SearchTask():

ImportError: dynamic module does not define module export function (PyInit_pylibvw)

Surprise library

I would love to see the surprise package available in the env. It seems to only rely on numpy (?), so it should be another pip install one-liner in the image.

I have a notebook application in mind for it.

pandas 21.0 -- fails reading lot of categorical values

Would you guys mind downgrading pandas to last stable version that is 20.3 instead of 21.0 -- lot of kernels are failing because of pandas-dev/pandas#18186

Dont know when they will fix this.

Please add package SMOTE

HI,

Please cn you add this package : https://github.com/scikit-learn-contrib/imbalanced-learn

Thanks

Authentication required when performing `docker pull`

Running docker pull kaggle/python on Arch Linux, I get the following output:

$ docker pull kaggle/python
Using default tag: latest
latest: Pulling from kaggle/python
8ad8b3f87b37: Pull complete
fb691515f399: Pull complete
6c3051db0635: Download complete
66faddd8f0d6: Download complete
3d7042b98994: Download complete
fe4d445aa40a: Download complete
7009d19096a4: Downloading 292.5MB/1.278GB
abfa5ac7621d: Download complete
90be25323b20: Downloading 51.88MB/1.618GB
d0c04a8d5747: Downloading
ca4c5048b9bb: Waiting
c80bb9c23c42: Waiting
unauthorized: authentication required

I believe I was able to successfully pull before (a week or so ago), but not today. How can I fix this? Thanks.

Installation of Folium librabry

I am not able to create a pull request, so I am creating an issue here. I want to use folium package for map visualization for in python, (kaggle kernel), please get it added to your python docker.
Thanks,

Mahesh

terminalplot

terminalplot is handy when testing scripts to help see trends in parameters over time or iterations

Request documentation - list of packages by category

Hello and thanks for the great docker image.

It would be very useful if you can provide a separate list of all programs in the image broken down by category. For example:

neural network
visualization
out-of-core learning
natural language processing
computer vision
computational geometry
etc

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.