heroku-python / conda-buildpack Goto Github PK
View Code? Open in Web Editor NEW[DEPRECATED] Buildpack for Conda.
License: MIT License
[DEPRECATED] Buildpack for Conda.
License: MIT License
I tried to add OpenCV 2.4.11 to the conda-requirements.txt
for installing but I just keep getting this after git push heroku master
.
remote: -----> Installing dependencies using Conda
remote: Fetching package metadata .......
remote: Solving package specifications: .
remote:
remote:
remote: PackageNotFoundError: Package not found: '' Package missing in current linux-64 channels:
remote: - opencv 2.4.11
remote:
remote: You can search for packages on anaconda.org with
remote:
remote: anaconda search -t conda opencv
remote:
remote: You may need to install the anaconda-client command line client with
remote:
remote: conda install anaconda-client
remote:
remote: Note that you have pinned specs in /app/.heroku/miniconda/conda-meta/pinned:
remote:
remote: ['nomkl']
remote:
remote:
remote: ! Push rejected, failed to compile Python/Conda app.
remote:
remote: ! Push failed
remote: Verifying deploy...
remote:
remote: ! Push rejected to sample-cv-flask.
remote:
To https://git.heroku.com/sample-cv-flask.git
! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'https://git.heroku.com/sample-cv-flask.git'
But after I searched for OpenCV I found it for linux 64.
menpo/opencv | 2.4.11 | conda | linux-64, win-32, win-64, linux-32, osx-64
Iโm now using Python 3, so it would be great if there could be a way to change the Python version based on some setting.
I cannot connect to kernel. I keep getting
Connecting to kernel
Hi, sorry if this is the wrong place for it, I'm wondering if there's a place to specify the package channel to do the equivalent of:
conda install --channel USERNAME PACKAGE
many thanks, sarah bird
Discussed with the Continuum guys last week.
Conda now supports an environment specification file that can include both pip and conda packages and thereby replace the two files requirements.txt and conda-requirements.txt. An example of such a file can be found at https://raw.githubusercontent.com/memex-explorer/memex-explorer/master/environment.yml
This would also solve issue #11, as environment.yml files can specify channels.
@kennethreitz, would you accept a PR to support this, or do you think it would be better to do this as a separate conda-env-buildpack?
On the second push to an app with the anaconda buildpack, I get the following error msg:
$ git push heroku-test master
Counting objects: 3, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (2/2), 257 bytes | 0 bytes/s, done.
Total 2 (delta 1), reused 0 (delta 0)
-----> Fetching custom git buildpack... done
-----> Python/Anaconda app detected
-----> Installing dependencies using Conda (1.4.6)
conda: error: package 'bitarray' version 0.8.1 is already installed. Supply a new version to install (e.g., 'bitarray=x.y.z'), or see 'conda update -h' for help on updating
! Push rejected, failed to compile Python/Anaconda app
Since the upgrade described here (Feb 5 2016): https://www.continuum.io/blog/developer-blog/anaconda-25-release-now-mkl-optimizations, conda is defaulting to the mkl optimized numpy and scipy, which require the ~120 MB mkl package. This can easily bump the slug size over 300 MB. It's simple to work around this by specifying "nomkl" in conda-requirements.txt, but perhaps that should be the default for this buildpack.
Nice to see this. I have a suggestion. At the end of the README, you note that Heroku has a relatively small filesize cap. You can run conda clean -pt
to delete all tarballs, and also all unused packages (there may be some from the Miniconda installers if newer versions of the packages are available). That should clear up a good deal of space.
We had some trouble integrating conda-buildpack with our Django app running on Heroku. The problem is that our app requires pylibmc
package which cannot be installed due to missing libmemcached
library. Using conda-buildpack together with of heroku-buildpack-python (I believe, you are maintaining it), obviously did not help, since conda-buildpack repeats pip install -r requirements.txt
and still fails when installing pylibmc
. We solved the problem by creating a fork of conda-buildpack where we
pip install -r requirements.txt
stepPYTHONPATH
to '/app/.heroku/miniconda/lib/python2.7/site-packages:$PYTHONPATH'
Wouldn't it make more sense to use conda-buildpack merely for installing Conda packages and leaving the Python part to heroku-buildpack-python?
I wrote already in the Heroku forum, but I thought to post an issue as my problem is actually quite specific to conda-buildpack
.
I need to run numpy
, scipy
and sklearn
on Python 3.4. However, the deployment does not succeed because the size limit of 300MB is exceeded by the slug (354MB).
This is what the conda-requirements.txt
look like:
conda=3.15.1=py34_0
conda-env=2.3.0=py34_0
python=3.4.3=0
pip=7.1.0=py34_0
setuptools=18.0.1=py34_0
openssl=1.0.1k=1
# what is really required
numpy=1.9.2=py34_0
scikit-learn=0.16.1=np19py34_0
scipy=0.16.0=np19py34_0
As the above leads to over 300MB, is there something I can get rid of? Is there a better approach to use Python 3.4 and these packages?
I recently switched to using the conda buildpack and now my app no longer serves static files correctly (404 responses for static assets). I'm using whitenoise to serve static files. Any ideas about how I could troubleshoot this?
I'm beginner with flask and trying to deploy a flask app to heroku using conda-buildpack and I'm getting the following error. Do you have any idea how I could fix it?
Mehrnooshs-MacBook-Pro-8:challenge mehrnoosh$ git push heroku master
Counting objects: 3, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 280 bytes | 0 bytes/s, done.
Total 3 (delta 2), reused 0 (delta 0)
remote: Compressing source files... done.
remote: Building source:
remote:
remote: -----> Python/Conda app detected
remote: -----> Preparing Python/Miniconda Environment
remote: PREFIX=/app/.heroku/miniconda
remote: installing: _cache-0.0-py27_x0 ...
remote: installing: python-2.7.11-0 ...
remote: installing: conda-env-2.4.5-py27_0 ...
remote: installing: openssl-1.0.2g-0 ...
remote: installing: pycosat-0.6.1-py27_0 ...
remote: installing: pyyaml-3.11-py27_1 ...
remote: installing: readline-6.2-2 ...
remote: installing: requests-2.9.1-py27_0 ...
remote: installing: sqlite-3.9.2-0 ...
remote: installing: tk-8.5.18-0 ...
remote: installing: yaml-0.1.6-0 ...
remote: installing: zlib-1.2.8-0 ...
remote: installing: conda-4.0.5-py27_0 ...
remote: installing: pycrypto-2.6.1-py27_0 ...
remote: installing: pip-8.1.1-py27_1 ...
remote: installing: wheel-0.29.0-py27_0 ...
remote: installing: setuptools-20.3-py27_0 ...
remote: Python 2.7.11 :: Continuum Analytics, Inc.
remote: creating default environment...
remote: installation finished.
remote: Fetching package metadata: ....
remote: Solving package specifications: .........
remote:
remote: Package plan for installation in environment /app/.heroku/miniconda:
remote:
remote: The following packages will be downloaded:
remote:
remote: package | build
remote: ---------------------------|-----------------
remote: conda-env-2.6.0 | 0 502 B
remote: sqlite-3.13.0 | 0 4.0 MB
remote: python-2.7.12 | 1 12.1 MB
remote: enum34-1.1.6 | py27_0 53 KB
remote: ruamel_yaml-0.11.14 | py27_0 352 KB
remote: conda-4.2.9 | py27_0 360 KB
remote: pip-8.1.2 | py27_0 1.5 MB
remote: ------------------------------------------------------------
remote: Total: 18.3 MB
remote:
remote: The following NEW packages will be INSTALLED:
remote:
remote: enum34: 1.1.6-py27_0
remote: ruamel_yaml: 0.11.14-py27_0
remote:
remote: The following packages will be UPDATED:
remote:
remote: conda: 4.0.5-py27_0 --> 4.2.9-py27_0
remote: conda-env: 2.4.5-py27_0 --> 2.6.0-0
remote: pip: 8.1.1-py27_1 --> 8.1.2-py27_0
remote: python: 2.7.11-0 --> 2.7.12-1
remote: sqlite: 3.9.2-0 --> 3.13.0-0
remote:
remote: Fetching packages ...
remote: conda-env-2.6. 100% |###############################| Time: 0:00:00 621.29 kB/s
remote: sqlite-3.13.0- 100% |###############################| Time: 0:00:00 38.11 MB/s
remote: python-2.7.12- 100% |###############################| Time: 0:00:00 40.53 MB/s
remote: enum34-1.1.6-p 100% |###############################| Time: 0:00:00 22.17 MB/s
remote: ruamel_yaml-0. 100% |###############################| Time: 0:00:00 26.14 MB/s
remote: conda-4.2.9-py 100% |###############################| Time: 0:00:00 10.70 MB/s
remote: pip-8.1.2-py27 100% |###############################| Time: 0:00:00 31.65 MB/s
remote: Extracting packages ...
remote: [ COMPLETE ]|##################################################| 100%
remote: Unlinking packages ...
remote: [ COMPLETE ]|##################################################| 100%
remote: Linking packages ...
remote: [ COMPLETE ]|##################################################| 100%
remote: added pinned file in /app/.heroku/miniconda/conda-meta/pinned
remote: Fetching package metadata .......
remote: Solving package specifications: ..........
remote:
remote: Package plan for installation in environment /app/.heroku/miniconda:
remote:
remote: The following packages will be downloaded:
remote:
remote: package | build
remote: ---------------------------|-----------------
remote: nomkl-1.0 | 0 402 B
remote:
remote: The following NEW packages will be INSTALLED:
remote:
remote: nomkl: 1.0-0
remote:
remote: Proceed ([y]/n)?
remote: Fetching packages ...
remote: nomkl-1.0-0.ta 100% |###############################| Time: 0:00:00 496.35 kB/s
remote: Extracting packages ...
remote: [ COMPLETE ]|##################################################| 100%
remote: Linking packages ...
remote: [ COMPLETE ]|##################################################| 100%
remote: -----> Installing dependencies using Conda
remote: Fetching package metadata .......
remote: .Solving package specifications: .
remote: PackageNotFoundError: Package not found: '' Packages missing in current linux-64 channels:
remote: - appscript 1.0.1 py27_0
remote: - launcher 1.0.0 3
remote: - libxslt 1.1.28 2
remote: - pyaudio 0.2.7 py27_0
remote: - python.app 1.2 py27_4
remote: - xlwings 0.3.5 py27_0
remote: - appnope 0.1.0 py27_0
remote: - netcdf4 1.2.4 np111py27_1
remote: - notebook 4.1.0 py27_2
remote:
remote: Close matches found; did you mean one of these?
remote:
remote: pyaudio: portaudio, pycairo
remote: python.app: python-snappy, python-ldap, python
remote:
remote: You can search for packages on anaconda.org with
remote:
remote: anaconda search -t conda xlwings
remote:
remote: (and similarly for the other packages)
remote:
remote: You may need to install the anaconda-client command line client with
remote:
remote: conda install anaconda-client
remote:
remote: Note that you have pinned specs in /app/.heroku/miniconda/conda-meta/pinned:
remote:
remote: ['nomkl']
remote:
remote:
remote: ! Push rejected, failed to compile Python/Conda app.
remote:
remote: ! Push failed
remote: Verifying deploy...
remote:
remote: ! Push rejected to aqueous-lake-38159.
remote:
To https://git.heroku.com/aqueous-lake-38159.git
! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'https://git.heroku.com/aqueous-lake-38159.git'
conda-requirements.txt
requirements.txt
This may be a dumb question, but how come this buildpack wasn't forked from the official heroku python buildpack?
This is what I get from Heroku when I try to deploy with this buildpack:
-----> Python/Conda app detected
-----> Preparing Python/Miniconda Environment
bash: Miniconda-latest-Linux-x86_64.sh: No such file or directory
! Push rejected, failed to compile Python/Conda app.
! Push failed
Traceback (most recent call last):
File "/app/.heroku/miniconda/bin/conda", line 3, in <module>
from conda.cli import main
ImportError: No module named 'conda'
See https://travis-ci.org/wei2912/bce-simulation/builds/44591616.
(It is also mentioned in #7 ),
The above error will arise if you use conda-buildpack with Python3.
But you can avoid this error by adding the below statements in conda-requirements.txt
(the versions depends on your environment).
conda=3.10.1=py34_0
conda-env=2.1.4=py34_0
requests=2.6.0=py34_0
So this error will be caused by Python2 conda
that is initially installed in the deploy process.
I think we have to consider the runtime.txt
to choose the conda version (Python2 or Python3).
Trying to deploy to heroku with scipy==0.14.0
in the conda-requirements.txt file yields this error:
gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DUSE__THREAD -I/usr/include/ffi -I/usr/include/libffi -I/app/.heroku/miniconda/include/python2.7 -c c/_cffi_backend.c -o build/temp.linux-x86_64-2.7/c/_cffi_backend.o
c/_cffi_backend.c:15:17: fatal error: ffi.h: No such file or directory
include <ffi.h>
^
compilation terminated.
error: command 'gcc' failed with exit status 1
The issue came up in the following SO question too: http://stackoverflow.com/questions/35775048/scipy-on-heroku-with-conda-failing-to-deploy
When trying to use your buildpack on bluemix it fails with this error:
----->Downloaded app package (44K)
-----> Downloaded app buildpack cache (215M)
Cloning into '/tmp/buildpacks/conda-buildpack'...
/tmp/buildpacks/conda-buildpack/bin/steps/conda_compile: line 10: /home/vcap/.heroku/miniconda/conda-meta/pinned: No such file or directory
Staging failed: Buildpack compilation step failedFAILED
BuildpackCompileFailed
i have forked your repository and reverted to commit ff54da2 and now it works.
Although Docker is better than using this Buildpack for Conda, because I did not want to introduce more complexity to the micro-project I was developing, I needed to use pip
.
The problem is that in
conda-buildpack/bin/steps/conda_compile
Line 22 in 87945f3
--allow-all-externals
is deprecated in modern pip versions (>18.0).
So I deleted it, and the build worked in Heroku. Hope I could help somebody :)
remote: cp: cannot stat '/app/!(tmp)': No such file or directory
remote: ! Push rejected, failed to compile Python/Conda app.
The conda build suddenly started failing today with the above error message. My whole website is down. Nothing has changed on the configs of my Heroku app.
Could you please help? @ken-reitz @jwkvam @yasyf @pnpnpn
Many thanks!
Here's my conda-requirements.txt
pip=8.0.1=py34_0
setuptools=19.4=py34_0
conda=3.19.0=py34_0
conda-env=2.4.5=py34_0
requests=2.9.1=py34_0
numpy=1.9.2=py34_0
python=3.5.1=0
scipy=0.16.0=np19py34_0
I get this error while building:
-----> Installing dependencies using Pip
Traceback (most recent call last):
File "/app/.heroku/miniconda/bin/pip", line 4, in <module>
import pip
ImportError: No module named 'pip'
! Push rejected, failed to compile Python/Conda app
1.9.2-py 51% |############### | Time: 0:00:00 27.04 MB/s
requests-2.9.1 0% | | ETA: --:--:-- 0.00 B/s
setuptools-19. 0% | | ETA: --:--:-- 0.00 B/s
conda-3.19.0-p 0% | | ETA: --:--:-- 0.00 B/s
pip-8.0.1-py34 0% | | ETA: --:--:-- 0.00 B/s
######################## | Time: 0:00:00 28.97 MB/s
scipy-0.16.0-n 0% | | ETA: --:--:-- 0.00 B/s
00:00 38.30 MB/s
| Time: 0:00:00 39.53 MB/s
s
| Time: 0:00:00 40.06 MB/s
81% |######################### | Time: 0:00:00 40.47 MB/s
00:00 40.79 MB/s
Extracting packages ...
[ ]| | 0%
Unlinking packages ...
[ COMPLETE ]| | 0%
Linking packages ...
[ COMPLETE ]| | 0%
-----> Installing dependencies using Pip
Traceback (most recent call last):
File "/app/.heroku/miniconda/bin/pip", line 4, in <module>
import pip
ImportError: No module named 'pip'
! Push rejected, failed to compile Python/Conda app
######################## | Time: 0:00:00 28.97 MB/s
scipy-0.16.0-n 0% | | ETA: --:--:-- 0.00 B/s
00:00 38.30 MB/s
| Time: 0:00:00 39.53 MB/s
s
| Time: 0:00:00 40.06 MB/s
81% |######################### | Time: 0:00:00 40.47 MB/s
00:00 40.79 MB/s
Extracting packages ...
[ ]| | 0%
Unlinking packages ...
[ COMPLETE ]| | 0%
Linking packages ...
[ COMPLETE ]| | 0%
-----> Installing dependencies using Pip
Traceback (most recent call last):
File "/app/.heroku/miniconda/bin/pip", line 4, in <module>
import pip
ImportError: No module named 'pip'
! Push rejected, failed to compile Python/Conda app
00:00 38.30 MB/s
| Time: 0:00:00 39.53 MB/s
s
| Time: 0:00:00 40.06 MB/s
81% |######################### | Time: 0:00:00 40.47 MB/s
00:00 40.79 MB/s
Extracting packages ...
[ ]| | 0%
Unlinking packages ...
[ COMPLETE ]| | 0%
Linking packages ...
[ COMPLETE ]| | 0%
-----> Installing dependencies using Pip
Traceback (most recent call last):
File "/app/.heroku/miniconda/bin/pip", line 4, in <module>
import pip
ImportError: No module named 'pip'
! Push rejected, failed to compile Python/Conda app
| Time: 0:00:00 39.53 MB/s
s
| Time: 0:00:00 40.06 MB/s
81% |######################### | Time: 0:00:00 40.47 MB/s
00:00 40.79 MB/s
Extracting packages ...
[ ]| | 0%
Unlinking packages ...
[ COMPLETE ]| | 0%
Linking packages ...
[ COMPLETE ]| | 0%
-----> Installing dependencies using Pip
Traceback (most recent call last):
File "/app/.heroku/miniconda/bin/pip", line 4, in <module>
import pip
ImportError: No module named 'pip'
! Push rejected, failed to compile Python/Conda app
s
| Time: 0:00:00 40.06 MB/s
81% |######################### | Time: 0:00:00 40.47 MB/s
00:00 40.79 MB/s
Extracting packages ...
[ ]| | 0%
Unlinking packages ...
[ COMPLETE ]| | 0%
Linking packages ...
[ COMPLETE ]| | 0%
-----> Installing dependencies using Pip
Traceback (most recent call last):
File "/app/.heroku/miniconda/bin/pip", line 4, in <module>
import pip
ImportError: No module named 'pip'
! Push rejected, failed to compile Python/Conda app
| Time: 0:00:00 40.06 MB/s
81% |######################### | Time: 0:00:00 40.47 MB/s
00:00 40.79 MB/s
Extracting packages ...
[ ]| | 0%
Unlinking packages ...
[ COMPLETE ]| | 0%
Linking packages ...
[ COMPLETE ]| | 0%
-----> Installing dependencies using Pip
Traceback (most recent call last):
File "/app/.heroku/miniconda/bin/pip", line 4, in <module>
import pip
ImportError: No module named 'pip'
! Push rejected, failed to compile Python/Conda app
81% |######################### | Time: 0:00:00 40.47 MB/s
00:00 40.79 MB/s
Extracting packages ...
[ ]| | 0%
Unlinking packages ...
[ COMPLETE ]| | 0%
Linking packages ...
[ COMPLETE ]| | 0%
-----> Installing dependencies using Pip
Traceback (most recent call last):
File "/app/.heroku/miniconda/bin/pip", line 4, in <module>
import pip
ImportError: No module named 'pip'
! Push rejected, failed to compile Python/Conda app
00:00 40.79 MB/s
Extracting packages ...
[ ]| | 0%
Unlinking packages ...
[ COMPLETE ]| | 0%
Linking packages ...
[ COMPLETE ]| | 0%
-----> Installing dependencies using Pip
Traceback (most recent call last):
File "/app/.heroku/miniconda/bin/pip", line 4, in <module>
import pip
ImportError: No module named 'pip'
! Push rejected, failed to compile Python/Conda app
How do I get around this?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.