machine-learning.ipynb on http://pangeo.pydata.org RuntimeError about pangeo-example-notebooks HOT 10 OPEN

@TomAugspurger, does this mean anything to you, perhaps a joblib/sklearn release schedule thing?

from pangeo-example-notebooks.

Did you import dask_ml.joblib, or import distributed.joblib first?

from pangeo-example-notebooks.

The imports (in order) throughout the notebook are:

from dask_kubernetes import KubeCluster
from dask.distributed import Client, progress
import dask_ml.joblib  # register the distriubted backend
from sklearn.datasets import make_classification
from sklearn.svm import SVC
from sklearn.model_selection import GridSearchCV
import pandas as pd
from sklearn.externals import joblib

from pangeo-example-notebooks.

Thanks, that would have raised a different error anyway.

Will take a look later.

from pangeo-example-notebooks.

Seeing same issue in example notebooks

from pangeo-example-notebooks.

Hopefully fixed by pangeo-data/helm-chart#51

You could maybe work around it by adding dask-ml to the worker-template.yaml, something like

    env:
      - name: EXTRA_CONDA_PACKAGES
        value: dask-ml

for now, but that isn't a long-term solution.

from pangeo-example-notebooks.

This machine learning notebook is working fine on our http://pangeo.esipfed.org instance using this Dockerfile based solely on conda-forge.

from pangeo-example-notebooks.

from pangeo-example-notebooks.

Ah, of course my diagnosis is incorrect, since the example doesn't actually require dask-ml, just scikit-learn and distributed.

I'll do some further debugging...

from pangeo-example-notebooks.

@TomAugspurger, we actually are using the notebook image for the workers too, so that old worker Dockerfile is misleading. The notebook environment contains dask-ml, which is required by the example notebook.

from pangeo-example-notebooks.