moneymeets / python-poetry-buildpack Goto Github PK
View Code? Open in Web Editor NEWHeroku buildpack that makes it possible to use Poetry with the official Python buildpack
License: MIT License
Heroku buildpack that makes it possible to use Poetry with the official Python buildpack
License: MIT License
Hi. My heroku build crashes with the following error:
-----> Export Python version from Poetry to Heroku runtime.txt file
-----> ^3.7 not valid, please specify an exact Python version (e.g. 3.8.1) in your pyproject.toml (and thus poetry.lock)
My pyproject.toml
contains a valid python version specification, following the recommendations in the Poetry documentation. According to the Poetry docs the ^
syntax should be valid.
[tool.poetry.dependencies]
python = "^3.7"
I am not exactly sure what I am doing wrong? Do I have the wrong Python version specification/different attribute name in pyproject.toml
? Any pointers, why it is complaining? Thanks.
Hello! It appears that Python version extraction is currently breaking if the python-versions = "x.x.x"
line in poetry.lock
includes a trailing space. This trailing space will be added if the Python version specification in pyproject.toml
also includes a trailing space. I found this because the VSCode TOML plugin I am using adds these spaces: oovm/vscode-toml#7.
When that happens, the Heroku build fails with the following message:
-----> 3.9.11 is not valid, please specify an exact Python version (e.g. 3.8.1) in your pyproject.toml (and thus poetry.lock)
To make this more robust, I propose that we strip out spaces in this command:
python-poetry-buildpack/bin/compile
Line 85 in 868d88b
Hello - thanks for this buildpack, we are successfully using it in production!
My issue: I'd like to have a dependency that's only installed in production, and not in development. In particular, it's recommended to use psycopg2-binary
in development and build psycopg2
from source for production.
Based on this issue it seems that the Poetry recommendation for this use case is to install the prod dependency as an extra
and then do poetry install/export -E production
when you want to install the prod version.
My ideal solution would be to set an environment variable in Heroku that sets the -E
in for the export step in this buildpack. If this is something you'd be willing to accept I can take a stab at implementing and open a PR, let me know!
Now that we are featured on Heroku, it would be nice to have a README:
https://elements.heroku.com/buildpacks/moneymeets/python-poetry-buildpack
Here is an official Heroku Python buildpack README for some inspiration:
https://elements.heroku.com/buildpacks/heroku/heroku-buildpack-python
GET_POETRY_IGNORE_DEPRECATION=1 doesn't help as well
-----> Building on the Heroku-22 stack
-----> Using buildpacks:
1. https://github.com/moneymeets/python-poetry-buildpack.git
2. heroku/python
3. https://github.com/heroku/heroku-buildpack-cli
-----> Python Poetry app detected
-----> Using Poetry version from POETRY_VERSION config var: 1.1.13
-----> Generate requirements.txt with Poetry
-----> Install Poetry
Retrieving Poetry metadataThis installer is deprecated, and cannot install Poetry 1.2.0a1 or newer. Additionally, Poetry installations created by this installer cannot `self update` to 1.2.0a1 or later. You should migrate to https://install.python-poetry.org instead. Instructions are available at https://python-poetry.org/docs/#installation. This installer will now exit. If you understand the above warning and wish to proceed anyway, set GET_POETRY_IGNORE_DEPRECATION=1 in the environment and run this installer again.
-----> Export requirements.txt from Poetry
/tmp/codon/tmp/buildpacks/6e1a99cb61ad645328dd1c64bd71597add4e2b5b/bin/compile: line 66: poetry: command not found
! Push rejected, failed to compile Python Poetry app.
! Push failed
Typically you'd want to specify your python version like:
python = '>=3.7,<4'
or perhaps more reasonably:
python = '^3.7.6'
For a new project that intends to support Python 3.7+. However nether of these options work and will fail with an error like:
remote: -----> Export Python version from Poetry to Heroku runtime.txt file
remote: -----> '^3.7.6' not valid, please specify an exact Python version (e.g. 3.8.1) in your pyproject.toml (and thus poetry.lock)
It would be nice if the latest supported version of python, given the specification string, were automatically picked.
My poetry files are not at the root of my project.
So, when I try to deploy the project on Heroku, it fails on the detect phase.
As a workaround, I noticed I can create symbolic links, it works but it's a bit ugly to me.
Is there any other way? I noticed a buildpack works with BUILD_DIR
var but I did not find where I can specify it.
Thanks.
Current Poetry installer gives
This installer is deprecated. Poetry versions installed using this script will not be able to use 'self update' command to upgrade to 1.2.0a1 or later.
warning. Related discussion can be found here
I am trying to use this buildpack for my poetry project but it fails when trying to fetch the custom buildpack:
-----> Building on the Heroku-22 stack
-----> Using buildpack: moneymeets/python-poetry-buildpack
! error fetching custom buildpack https://buildpack-registry.s3.amazonaws.com/buildpacks/moneymeets/python-poetry-buildpack.tgz
! Push failed
I am not familiar with heroku or this buildpack.
Would appreciate if someone could point me to a solution please πΊ
I have also reached out to Heroku support
After Poetry upgrade my heroku deployment stopped working (deploy log attached), switching to previous version with heroku config:set POETRY_VERSION=1.0.10
works as expected:
Here is the output with the latest version:
remote: -----> Build succeeded!
remote: -----> Python Poetry app detected
remote: -----> No Poetry version specified in POETRY_VERSION config var. Defaulting to 1.1.0.
remote: -----> Generate requirements.txt with Poetry
remote: -----> Install Poetry
remote: Retrieving Poetry metadata
remote:
remote: # Welcome to Poetry!
remote:
remote: This will download and install the latest version of Poetry,
remote: a dependency and package manager for Python.
remote:
remote: It will add the `poetry` command to Poetry's bin directory, located at:
remote:
remote: $HOME/.poetry/bin
remote:
remote: This path will then be added to your `PATH` environment variable by
remote: modifying the profile file located at:
remote:
remote: $HOME/.profile
remote:
remote: You can uninstall at any time by executing this script with the --uninstall option,
remote: and these changes will be reverted.
remote:
remote: Installing version: 1.1.0
remote: - Downloading poetry-1.1.0-linux.tar.gz (57.00MB)
remote:
remote: Poetry (1.1.0) is installed now. Great!
remote:
remote: To get started you need Poetry's bin directory ($HOME/.poetry/bin) in your `PATH`
remote: environment variable. Next time you log in this will be done
remote: automatically.
remote:
remote: To configure your current shell run `source $HOME/.poetry/env`
remote:
remote: -----> Export requirements.txt from Poetry
remote: /app/.poetry/lib/poetry/_vendor/py2.7/subprocess32.py:149: RuntimeWarning: The _posixsubprocess module is not being used. Child process reliability may suffer if your program uses threads.
remote: "program uses threads.", RuntimeWarning)
remote:
remote: Python 2.7 will no longer be supported in the next feature release of Poetry (1.2).
remote: You should consider updating your Python version to a supported one.
remote:
remote: Note that you will still be able to manage Python 2.7 projects by using the env command.
remote: See https://python-poetry.org/docs/managing-environments/ for more information.
remote:
remote:
remote: IndexError
remote:
remote: list index out of range
remote: -----> Export Python version from Poetry to Heroku runtime.txt file
remote: -----> Write 3.8.5 into runtime.txt
remote: -----> App not compatible with buildpack: https://buildpack-registry.s3.amazonaws.com/buildpacks/heroku/python.tgz
remote: More info: https://devcenter.heroku.com/articles/buildpacks#detection-failure
remote:
remote: ! Push failed
remote: Verifying deploy...
remote:
remote: ! Push rejected to heroku-example-app.
remote:
Hello and thank you for this useful buildpack!
We hit dependabot/dependabot-core#6462 in our project, where Dependabot generates a Python version with the tilde specifier (~3.11
) in poetry.lock
when pyproject.toml
contains a plain full version like 3.11.1
, which is considered invalid with this buildpack.
I saw the discussion in #6, but would you consider supporting at least the exact version specifier (==3.11.1
) as suggested in the Dependabot issue discussion? That's what Dependabot generates if it's used in pyproject.toml
.
I'm trying to set the POETRY_VERSION to 1.6.1
however, heroku client just resets it to 1.1.13
to replicate run:
heroku config:set POETRY_VERSION=1.6.1
expected response:
POETRY_VERSION: 1.6.1
actual response:
POETRY_VERSION: 1.1.13
Is there a mechanism that prevents updating this env?
Since poetry currently does not export runtime.txt
Heroku installs a different Python version than what is specified in pyproject.toml.
Hello dear @marns93 ,
would you mind publishing the pack to Buildpacks.io?
Here is a quick guide : https://buildpacks.io/docs/buildpack-author-guide/publish-a-buildpack/
heroku/python
to deploy a Python project to Heroku.poetry
commands in the CI checks:.local/bin/poetry check
.local/bin/poetry lock --no-update --check
[tool.poetry.dependencies]
python = "3.11.1"
Poetry installs and generates a requirements.txt and runtime.txt, Heroku installs Python version 3.11.1 and the requirements.txt packages, Poetry's checks succeed.
Poetry installs and generates a requirements.txt and runtime.txt, Heroku installs Python version 3.11.1 and the requirements.txt packages, Poetry's checks fail with an error like this:
Python path configuration:
PYTHONHOME = '/app/.heroku/python'
PYTHONPATH = '/app'
program name = '/app/.local/share/pypoetry/venv/bin/python'
isolated = 0
environment = 1
user site = 1
import site = 1
sys._base_executable = '/app/.local/share/pypoetry/venv/bin/python'
sys.base_prefix = '/app/.heroku/python'
sys.base_exec_prefix = '/app/.heroku/python'
sys.platlibdir = 'lib'
sys.executable = '/app/.local/share/pypoetry/venv/bin/python'
sys.prefix = '/app/.heroku/python'
sys.exec_prefix = '/app/.heroku/python'
sys.path = [
'/app',
'/app/.heroku/python/lib/python310.zip',
'/app/.heroku/python/lib/python3.10',
'/app/.heroku/python/lib/python3.10/lib-dynload',
]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'
Current thread 0x00007ff6ee928480 (most recent call first):
<no Python frame>
Poetry is installed into its venv using the Heroku default Python for heroku-22
which is 3.10.9. Once heroku/python
switches to 3.11.1, the 3.10.9 installation is deleted, which breaks the Poetry venv.
In this buildpack, use pyenv
to install an isolated Python that Poetry can use and rely on. Related to the discussion here: heroku/heroku-buildpack-python#796 (comment)
I'm not sure how to do this in a way that plays nicely with Heroku's caching and stuff. I haven't made a buildpack of my own before.
Do not upgrade to 3.11.
Hello,
I have forked your repo and modified it as such as to have an action which deploys the buildpack to buildpacks.io when creating a release.
Would you be interested in merging this? I'd create a PR if yes
What are your thoughts on having this buildpack run poetry lock
if it doesn't find an existing poetry.lock
file?
I have dependabot running on my GitHub repo. Each time I merge a dependabot PR, it forces a cascade rebase of all other dependency PRs because of lockfile hash conflicts. It'd save me a ton of time (and CI/CD credits) if I could not commit the lockfile and just have it be generated at runtime.
I tried to deploy my app that uses Poetry, but got an error:
-----> Patch Poetry export file
-----> Export requirements.txt from Poetry
AttributeError
'NoneType' object has no attribute 'to_dependency'
at ~/.poetry/lib/poetry/packages/locker.py:234 in get_project_dependencies
230β # project level dependencies take precedence
231β continue
232β
233β # we make a copy to avoid any side-effects
β 234β requirement = deepcopy(__get_locked_package(requirement).to_dependency())
235β requirement._category = pkg.category
236β
237β if pinned_versions:
238β requirement.set_constraint(
-----> Export Python version from Poetry to Heroku runtime.txt file
-----> Write 3.8.6 into runtime.txt
-----> App not compatible with buildpack: https://buildpack-registry.s3.amazonaws.com/buildpacks/heroku/python.tgz
More info: https://devcenter.heroku.com/articles/buildpacks#detection-failure
! Push failed
My poetry.lock
file:
[[package]]
category = "main"
description = "Fast and simple WSGI-framework for small web-applications."
name = "bottle"
optional = false
python-versions = "*"
version = "0.12.18"
[[package]]
category = "main"
description = "WSGI HTTP Server for UNIX"
name = "gunicorn"
optional = false
python-versions = ">=3.4"
version = "20.0.4"
[package.dependencies]
setuptools = ">=3.0"
[package.extras]
eventlet = ["eventlet (>=0.9.7)"]
gevent = ["gevent (>=0.13)"]
setproctitle = ["setproctitle"]
tornado = ["tornado (>=0.2)"]
[[package]]
category = "main"
description = "Python client for Redis key-value store"
name = "redis"
optional = false
python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*"
version = "3.5.3"
[package.extras]
hiredis = ["hiredis (>=0.1.3)"]
[metadata]
content-hash = "184994068e190b2720ed44423cc8673e8300ceae582e90301adbe7502e598b42"
lock-version = "1.0"
python-versions = "3.8.6"
[metadata.files]
bottle = [
{file = "bottle-0.12.18-py3-none-any.whl", hash = "sha256:43157254e88f32c6be16f8d9eb1f1d1472396a4e174ebd2bf62544854ecf37e7"},
{file = "bottle-0.12.18.tar.gz", hash = "sha256:0819b74b145a7def225c0e83b16a4d5711fde751cd92bae467a69efce720f69e"},
]
gunicorn = [
{file = "gunicorn-20.0.4-py2.py3-none-any.whl", hash = "sha256:cd4a810dd51bf497552cf3f863b575dabd73d6ad6a91075b65936b151cbf4f9c"},
{file = "gunicorn-20.0.4.tar.gz", hash = "sha256:1904bb2b8a43658807108d59c3f3d56c2b6121a701161de0ddf9ad140073c626"},
]
redis = [
{file = "redis-3.5.3-py2.py3-none-any.whl", hash = "sha256:432b788c4530cfe16d8d943a09d40ca6c16149727e4afe8c2c9d5580c59d9f24"},
{file = "redis-3.5.3.tar.gz", hash = "sha256:0e7e0cfca8660dea8b7d5cd8c4f6c5e29e11f31158c0b0ae91a397f00e5a05a2"},
]
Hello,
I use the latest version of this buildpack and since #66 it installs poetry-plugin-export
, but this plugin requires Poetry 1.6 and so the deployment doesnβt work if one uses an older version.
Enumerating objects: 32, done.
Counting objects: 100% (32/32), done.
Delta compression using up to 8 threads
Compressing objects: 100% (13/13), done.
Writing objects: 100% (17/17), 2.48 KiB | 636.00 KiB/s, done.
Total 17 (delta 9), reused 0 (delta 0), pack-reused 0
-----> Cleaning up...
-----> Building myapp from herokuish
-----> Adding BUILD_ENV to build environment...
BUILD_ENV added successfully
-----> Multipack app detected
=====> Downloading Buildpack: https://github.com/moneymeets/python-poetry-buildpack.git
=====> Detected Framework: Python Poetry
-----> Read Poetry version 1.5.1 from poetry.lock
-----> Generate requirements.txt with Poetry
-----> Install Poetry
Retrieving Poetry metadata
# Welcome to Poetry!
This will download and install the latest version of Poetry,
a dependency and package manager for Python.
It will add the `poetry` command to Poetry's bin directory, located at:
/app/.local/bin
You can uninstall at any time by executing this script with the --uninstall option,
and these changes will be reverted.
Installing Poetry (1.5.1)
Installing Poetry (1.5.1): Creating environment
Installing Poetry (1.5.1): Installing Poetry
Installing Poetry (1.5.1): Creating script
Installing Poetry (1.5.1): Done
Poetry (1.5.1) is installed now. Great!
To get started you need Poetry's bin directory (/app/.local/bin) in your `PATH`
environment variable.
Add `export PATH="/app/.local/bin:$PATH"` to your shell configuration file.
Alternatively, you can call Poetry explicitly with `/app/.local/bin/poetry`.
You can test that everything is set up by executing:
`poetry --version`
-----> Add Poetry to the PATH
-----> Install poetry-plugin-export
remote:
remote: Because no versions of poetry-plugin-export match >1.6.0,<2.0.0
remote: and poetry-plugin-export (1.6.0) depends on poetry (>=1.6.0,<2.0.0), poetry-plugin-export (>=1.6.0,<2.0.0) requires poetry (>=1.6.0,<2.0.0).
remote: So, because poetry-instance depends on both poetry (1.5.1) and poetry-plugin-export (^1.6.0), version solving failed.
Using version ^1.6.0 for poetry-plugin-export
Updating dependencies
Resolving dependencies...
remote: ! Failure during app build
remote: ! Retagging old image 793abcde as myremote/myapp:latest
remote: ! App build failed
Would it be possible to install the plugin only if the Poetry version is >1.6?
What is the right way to use this buildback, while using Heroku Multi Procfile buildpack in the same time?
I tried this way:
My Python app (and Procfile) is in back/
folder of my root repo. Obviously, I soon as I tried to deploy, I get this error:
Any idea? workaround? Thanks.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.