Jupyter setup failure since April 5 about initialization-actions HOT 5 CLOSED

I got stuck at first on the ImportError: No module named google_compute_engine error since I couldn't figure out how it ever worked :)

Basically, the clobbering of PATH to have conda first always breaks gsutil, so the little gsutil stat trick trying to fetch notebooks from GCS just never actually worked (but also was untested and apparently unused).

But it's in a conditional so that's actually just a red herring. @nehalecky is correct that #103 should resolve this.

from initialization-actions.

Merged #103 - should work out of the box once again.

from initialization-actions.

@dennishuo, thanks for the details on that ImportError. Is gsutil the only component of glcoud sdk that doesn't play nice with conda, or are there others? Any more details on that would be appreciated as we're (@jeffkpayne) currently updating a Docker image that has both. 😄

Also, any more info on the use of Docker containers in Dataproc? :)

Thanks again!

from initialization-actions.

@nehalecky To be honest we don't have much coverage of compatibility constraints between cloud SDK and conda, though at least in my experience gsutil tends to be more picky than base gcloud commands. A cursory check seems to show gcloud still working okay with conda installed and overridden on PATH. I'll be happy to hear about your findings if you do discover interesting quirks though!

Re: Docker containers, have you played around with the datalab initialization action which uses a Docker container for the dev environment? It could be a worthwhile pattern to apply to other init actions.

I suppose the part that's less clear is whether there's a role for docker containers on worker nodes.

from initialization-actions.

Yep, that gsutil stat... trick was definitely not well tested and, for other reasons, the goal that it was supposed to provide was never really utilized by us anyway.

from initialization-actions.