support generated requirements .txt files about makefile.venv HOT 4 CLOSED

Thank you for reporting this issue! I was not familiar with pip-compile before and I have not anticipated such use case while creating Makefile.venv.

The task you're trying to do is twofold:

1. You want to be able to automatically generate 'requirements.txt' with a Makefile recipe
2. You want to use a tool from the virtual environment before virtual environment is finalized by Makefile.venv

First part was not possible (at least not easily doable) with Makefile.venv (v2020.05.07) because it allowed for REQUIREMENTS_TXT to reference non-existent files. No errors would be raised and no Makefile recipes would be checked to try to create the missing files, Makefile.venv would just move along happily. I've changed that today, the code is currently in a feature branch: issue8

Second part was and is doable but undocumented. I do not yet see how to describe it in generic terms, may be you will help?

Here is how it can be done for your case:

# Makefile

### This part was not possible with v2020.05.07

# You need to explicitly specify this value because by default it's ok for
# requirements.txt to be missing. This statement MUST come before `include
# Makefile.venv`
REQUIREMENTS_TXT=requirements.txt


### Everything below relies on features existing in v2020.05.07

include Makefile.venv


# Save pip-compile path to variable for brevity
# You may omit $(EXE) suffix if you're ok with Windows builds being broken
PIP_COMPILE=$(VENV)/pip-compile$(EXE)


# This and the next recipe MUST be defined after include statement.
# You need to inject pip-compile into virtual environment
# before Makefile.venv finishes working on it, but after venv is created.
# There already exists a target you can add as dependency for this case:
$(PIP_COMPILE): | $(VENV)
	$(VENV)/pip install pip-tools  # or whatever method you prefer
	touch $@


# Your requirements file directly depends upon *.in file and also requires
# pip-compile to be available
requirements.txt: requirements.in | $(PIP_COMPILE)
	$(PIP_COMPILE) --output-file $@ $<

And that's all. The rest of your Makefile should be written as usual when working with Makefile.venv: depend on venv, call $(VENV)/binaries, etc. The trick is to use $(VENV) as dependency for $(PIP_COMPILE) - that way we can avoid circular dependnecy and pip-compile will be installed after bare venv is created but before Makefile.venv attempts to fill it with packages from 'requirements.txt'. I've added a full working example to the test suite.

If nothing is broken and there is no new feedback in a couple of days I will merge the changes to master and will publish a new release.

PS: may I ask what project are you using Makefile.venv for?

from makefile.venv.

Thank you, we'll try it next week.

We're evaluating Makefile.venv for some internal GROOVE X projects.

from makefile.venv.

Thank you for the enhancement and detailed instructions-- it seems to be working!

from makefile.venv.

Good :) I've just pushed a new release, v2020.08.04, which includes these changes.

from makefile.venv.