basnijholt / unidep Goto Github PK

View Code? Open in Web Editor NEW

215.0 5.0 5.0 645 KB

Single source of truth with requirements for pip and conda

Home Page: https://unidep.readthedocs.io

License: BSD 3-Clause "New" or "Revised" License

Python 100.00%

conda conda-environment conda-lock pip pip-compile setuptools

unidep's People

Contributors

Stargazers

Watchers

Forkers

richardsonjf marcellaholm markplagge gabyto19 phosphoenix sbalk

unidep's Issues

Make YAML an optional dependency

Add `post_install_command` e.g., `yarn install`

Support extra optional dependencies

Add `environment.yaml` -> `requirements.yaml` conversion script

Allow specifying local dependencies that aren't managed by unidep in `local_dependencies`

When to use?

You prefer to install packages with conda but still want your package to be pip installable
You're tired of keeping your requirements.txt and environment.yaml in sync
You want a ultra low effort full development environment setup

Allow specifying both conda and pip for different platforms

-  conda: numpy # [linux]
   pip: numpy # [macos]

Might not work when combining environments because sel(platform): ... is not possible in pip:

Allow specifying `environment.yaml` in `local_dependencies`

Edit: while implementing this in #113 I realized this is a bad idea.
We cannot extract any dependencies that would also be installable with conda and therefore the whole situation becomes much more complicated.

Therefore the only option currently remains for unidep to be "infectious" for local dependencies.

(Bas): !=5 should be removed but this is not yet implemented # noqa: TD003, FIX002

        # TODO (Bas): !=5 should be removed but this is not yet implemented  # noqa: TD003, FIX002

unidep/tests/test_version_conflicts.py

Line 62 in 31894d9

([">1"], ">1"),

Allow `unidep install` from an URL (Git repo)

Document supported pinnings

FAQ question: can't conda already handle pip dependencies?

FAQ question: difference with conda-lock?

Use selectors in pyproject.toml

This means that it should detect the platform and select the pip packages

Add `--ignore-pin package-name` to `merge`, `install`, and `install-all`

Add command to install `conda-lock.yml` file and local dependencies

Test on Windows

Check strictyaml

https://github.com/crdoconnor/strictyaml

Convert selector comments to dict style `sel(platform): package-name` optionally

Only preserve selector comments

Right now all comments are preserved. Instead we should have a regex that finds [sel].

Implement `conda-lock` subcommand that creates one large lock-file and many smaller lock-files for the sub folders

Support flit

Use `grayskull_pypi_mapping.yaml`

https://github.com/regro/cf-graph-countyfair/blob/master/mappings/pypi/grayskull_pypi_mapping.yaml

Logos

Implement `install-all` subcommand for monorepos

FAQ

How is this different from conda/mamba/pip?
When to use this? Internal projects, in research and data science.
You prefer to install packages with conda but still want your package to be pip installable
You're tired of keeping your requirements.txt and environment.yaml in sync
You want a ultra low effort full development environment setup
I found a project using unidep, now what?

Support hatch

https://discuss.python.org/t/dynamic-metadata-populated-by-entry-points/36917/2

https://hatch.pypa.io/latest/plugins/metadata-hook/reference/#known-third-party

Only validate pinning if there are multiple different ones, otherwise leave them as is

Support poetry

Never allow PlatformNotIdentified error during pip install

FAQ: differences between hatch-conda and pdm-conda

From https://discuss.python.org/t/announcing-unidep-unified-conda-and-pip-dependency-management-via-pyproject-toml-or-requirements-yaml/42114/3?u=basnijholt:

Implement logic in selectors `and` and `or`

https://docs.conda.io/projects/conda-build/en/stable/resources/define-metadata.html#preprocessing-selectors

Consider renaming `includes` to `local_dependencies`

unidep install with --conda-env-name refers to prefix environment instead named environment

unidep install requirements.yaml --conda-env-name 311

ValueError: Could not find conda prefix with name `311`. Available prefixes:
👉 C:\ProgramData\mambaforge_22.9.0.2
👉 C:\Users\login\.conda\envs\310
👉 C:\Users\login\.conda\envs\311
👉 C:\Users\login\.conda\envs\312
👉 C:\Users\login\.conda\envs\39

It seems the named-environments fall back to computing prefix-environments but don't find them (?), despite 311 existing.
I'm running unidep from a pipx install.

Create `conda-forge` recipe

See conda-forge/staged-recipes#24632

Sort dependencies in generated environment.yaml

Convert logic operators for pins (e.g., conda uses `=` and pip `==`)

Anchor links to headers not working in RTD

Seems blocked by executablebooks/MyST-Parser#564

Question: will unidep handle building source distribution archives via pypa `build`?

I'm really enjoying this package, this is great and solves a lot of problems I've been struggling with. I should also note I'm new to python packaging and distribution. I noted recently that I have to move the dependencies out from behind the [tool.unidep] section in my pyproject.toml to be able to use pypa's build frontend. Is this something that unidep plans on providing an interface for?

Lines 479 to 520 in 9afbaad

 if not skip_local: 

 for file in files: 

 if is_pip_installable(file.parent): 

 installable.append(file.parent) 

 else: # pragma: no cover 

 print( 

 f"⚠️ Project {file.parent} is not pip installable. " 

 "Could not find setup.py or [build-system] in pyproject.toml.", 

 ) 

 if installable: 

 _pip_install_local( 

 *installable, 

 editable=editable, 

 dry_run=dry_run, 

 flags=pip_flags, 

 ) 

 if not skip_local: 

 # Install local dependencies (if any) included via `includes:` 

 local_dependencies = parse_project_dependencies( 

 *files, 

 check_pip_installable=True, 

 verbose=verbose, 

 ) 

 names = {k.name: [dep.name for dep in v] for k, v in local_dependencies.items()} 

 print(f"📝 Found local dependencies: {names}\n") 

 installed = {p.resolve() for p in installable} 

 deps = sorted( 

 { 

 dep 

 for deps in local_dependencies.values() 

 for dep in deps 

 if dep.resolve() not in installed 

 }, 

 ) 

 if deps: 

 _pip_install_local( 

 *deps, 

 editable=editable, 

 dry_run=dry_run, 

 flags=pip_flags, 

 )

Posts and featured

Posted by others

TLDR newsletter (2023-12-13)
PyCoder's Weekly: Issue #614 (2024-01-30)
Python Bytes Podcast: #366: Put It In The Backlog (2024-01-09) @ 6:57 - 11:19
Marktechpost (2024-02-10) (written using ChatGPT?)

Self posted

	if not skip_local:
	for file in files:
	if is_pip_installable(file.parent):
	installable.append(file.parent)
	else: # pragma: no cover
	print(
	f"⚠️ Project {file.parent} is not pip installable. "
	"Could not find setup.py or [build-system] in pyproject.toml.",
	)
	if installable:
	_pip_install_local(
	*installable,
	editable=editable,
	dry_run=dry_run,
	flags=pip_flags,
	)

	if not skip_local:
	# Install local dependencies (if any) included via `includes:`
	local_dependencies = parse_project_dependencies(
	*files,
	check_pip_installable=True,
	verbose=verbose,
	)
	names = {k.name: [dep.name for dep in v] for k, v in local_dependencies.items()}
	print(f"📝 Found local dependencies: {names}\n")
	installed = {p.resolve() for p in installable}
	deps = sorted(
	{
	dep
	for deps in local_dependencies.values()
	for dep in deps
	if dep.resolve() not in installed
	},
	)
	if deps:
	_pip_install_local(
	*deps,
	editable=editable,
	dry_run=dry_run,
	flags=pip_flags,
	)