Giter VIP home page Giter VIP logo

tox-gh-actions's Introduction

tox-gh-actions

PyPI version PyPI Supported Python Versions GitHub Actions (Tests) codecov

tox-gh-actions is a tox plugin which helps running tox on GitHub Actions with multiple different Python versions on multiple workers in parallel. This project is inspired by tox-travis.

Versions and Compatibility

Currently, tox-gh-actions supports both tox 3 and 4, but users need to install the appropriate version by following the table below.

tox tox-gh-actions Supported by tox-gh-actions Branch
4.x 3.x Yes (stable) master
3.x 2.x Yes (stable) tox3

Features

When running tox on GitHub Actions, tox-gh-actions

  • detects which environment to run based on configurations and
  • provides utilities such as grouping log lines.

Usage

  1. Add configurations under [gh-actions] section along with tox's configuration.

  2. Install tox-gh-actions package in the GitHub Actions workflow before running tox command.

Examples

Basic Example

The following configuration will create 4 jobs when running the workflow on GitHub Actions.

  • On Python 3.7 job, tox runs py37 environment
  • On Python 3.8 job, tox runs py38 environment
  • On Python 3.9 job, tox runs py39 environment
  • On Python 3.10 job, tox runs py310 and mypy environments

tox-gh-actions Configuration

Add [gh-actions] section to the same file as tox's configuration.

If you're using tox.ini:

[tox]
envlist = py37, py38, py39, py310, mypy

[gh-actions]
python =
    3.7: py37
    3.8: py38
    3.9: py39
    3.10: py310, mypy

[testenv]
...

If you're using setup.cfg:

[tox:tox]
envlist = py37, py38, py39, py310, mypy

[gh-actions]
python =
    3.7: py37
    3.8: py38
    3.9: py39
    3.10: py310, mypy

[testenv]
...

If you're using pyproject.toml:

[tool.tox]
legacy_tox_ini = """
[tox]
envlist = py37, py38, py39, py310, mypy

[gh-actions]
python =
    3.7: py37
    3.8: py38
    3.9: py39
    3.10: py310, mypy

[testenv]
"""

Workflow Configuration

.github/workflows/<workflow>.yml:

name: Python package

on:
  - push
  - pull_request

jobs:
  build:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ['3.7', '3.8', '3.9', '3.10']

    steps:
    - uses: actions/checkout@v3
    - name: Set up Python ${{ matrix.python-version }}
      uses: actions/setup-python@v4
      with:
        python-version: ${{ matrix.python-version }}
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        python -m pip install tox tox-gh-actions
    - name: Test with tox
      run: tox

Advanced Examples

Factor-Conditional Settings: Python Version

The following configuration will create 2 jobs when running the workflow on GitHub Actions.

  • On Python 3.7 job, tox runs py37-django22 and py37-django32 environments
  • On Python 3.8 job, tox runs py38-django32 environment

tox.ini:

[tox]
envlist = py37-django{22,32}, py38-django32

[gh-actions]
python =
    3.7: py37
    3.8: py38

[testenv]
...

When using pre-release versions of Python, please do not specify -beta or -dev in tox.ini.

.github/workflows/<workflow>.yml:

...
jobs:
  build:
    strategy:
      matrix:
        python-version: [3.9, 3.10.0-beta.3]
...

tox.ini:

[tox]
envlist = py39, py310

[gh-actions]
python =
    3.9: py39
    3.10: py310
    # The following won't work
    # 3.10-beta.3: py310
    # 3.10-dev: py310

[testenv]
...

PyPy is also supported in the python configuration key. Support of Pyston is experimental and not tested by our CI.

tox.ini:

[tox]
envlist = py37, py38, pypy3, pyston38

[gh-actions]
python =
    3.7: py37
    3.8: py38, mypy
    pypy-3.7: pypy3
    pyston-3.8: pyston38

[testenv]
...

[testenv:pyston38]
basepython = pyston38

You can also specify without minor versions in the python configuration key.

tox.ini:

[tox]
envlist = py3, pypy3

[gh-actions]
python =
    3: py3, mypy
    pypy-3: pypy3

[testenv]
...

If there are multiple matching Python versions in the configuration, only the most precise one is used. For example, if you are running CPython 3.8 and gh-actions.python has both 3 and 3.8, tox-gh-actions gets factors only from the key 3.8.

Changed in 3.0: pypy3 is not supported in the configuration anymore. Please use pypy-3 instead.

Factor-Conditional Settings: Environment Variable

You can also use environment variable to decide which environment to run. The following is an example to install different dependency based on platform. It will create 9 jobs when running the workflow on GitHub Actions.

  • On Python 3.8/ubuntu-latest job, tox runs py38-linux environment
  • On Python 3.9/ubuntu-latest job, tox runs py39-linux environment
  • and so on.

.github/workflows/<workflow>.yml:

name: Python package

on:
  - push
  - pull_request

jobs:
  build:
    runs-on: ${{ matrix.platform }}
    strategy:
      matrix:
        platform: [ubuntu-latest, macos-latest, windows-latest]
        python-version: ['3.8', '3.9', '3.10']

    steps:
    - uses: actions/checkout@v3
    - name: Set up Python ${{ matrix.python-version }}
      uses: actions/setup-python@v4
      with:
        python-version: ${{ matrix.python-version }}
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        python -m pip install tox tox-gh-actions
    - name: Test with tox
      run: tox
      env:
        PLATFORM: ${{ matrix.platform }}

tox.ini:

[tox]
envlist = py{38,39,310}-{linux,macos,windows}

[gh-actions]
python =
    3.8: py38
    3.9: py39
    3.10: py310

[gh-actions:env]
PLATFORM =
    ubuntu-latest: linux
    macos-latest: macos
    windows-latest: windows

[testenv]
deps =
  <common dependency>
  linux: <Linux specific deps>
  macos: <macOS specific deps>
  windows: <Windows specific deps>
...

Changed in 3.0: Environment variables should not use lowercase letters. Because of the limitation in tox's configuration loading API, tox-gh-actions always convert keys in [gh-actions:env] to uppercase.

tox requires

If your project uses tox's requires configuration, you must add tox-gh-actionsto therequires` configuration as well. Otherwise, tox-gh-actions won't be loaded as a tox plugin.

[tox]
requires =
  tox-conda
  tox-gh-actions

Overriding Environments to Run

Changed in 2.0: When a list of environments to run is specified explicitly via -e option or TOXENV environment variable (tox's help), tox-gh-actions respects the given environments and simply runs the given environments without enforcing its configuration.

Before 2.0, tox-gh-actions was always enforcing its configuration even when a list of environments is given explicitly.

Versioning

This project follows PEP 440 and uses a format of major.minor.patch (X.Y.Z). The major version (X) will be incremented when we make backward incompatible changes to a public API. The public API of this project is the configuration of tox-gh-actions. The major version can be also incremented when we require a new version of tox.

This project tries not to introduce backward incompatibles changes as much as possible so that users don't need to update their project's configuration too frequently.

tox-gh-actions 3.x may drop support of unsupported Python 3.y versions in the future without bumping its major version.

Understanding Behavior of tox-gh-actions

How tox-gh-actions Works

See ARCHITECTURE.md for more details.

Logging

tox-gh-actions writes log messages using the standard logging module. This is handy for understanding behavior of tox-gh-actions and for debugging tox-gh-actions. To see the log messages, please run tox -vv.

tox-gh-actions's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

tox-gh-actions's Issues

Individual steps for environments

This is more of a technical question, but would it be possible to present each environment in tox as a separate step in Actions? I have several environments for various things (documentation, tests, generated code, etc.) and if one of them fails, it's just presented as "tox failed" and I have to dig through the log to find what actually failed. Would be very neat if every environment got its own step. Not sure what is possible with the Actions API, but I almost assume this is not possible?

PyPy support

Do you have any plans for adding support for pypy2 and pypy3? This is the only thing that stops me from using tox-gh-actions right now.

Let me know if I can help you with that in any way!

Python 3.6 environment not found, but works for 3.7, 3,8, 3,9...

First of all, thank you for providing this excellent library! I don't take your work or help for granted! :)

Question
tox-gh-actions does not find my Python 3.6 environment. I'm getting the error "Warning: WARNING: tox-gh-actions couldn't find environments matching the provided factors from envlist. Please use tox -vv to get more detailed logs."

I have tried googling this, but I only find examples with configs similar to mine. Also, since the other environments work, I really struggle to understand how 3.6 is different. Would you mind helping me getting my tests for 3.6 run on GitHub Actions?

Context
This is for an open source-project, so the full run, with all config is available here:
https://github.com/EmilStenstrom/django-components/actions/runs/4518371119/jobs/7958205977

Researching this bug I found the fail_on_no_env option, which explains why I missed that we didn't have any tests for 3.6 run at all. Would much prefer to have this as the default. Here's the same run with this flag enabled:
https://github.com/EmilStenstrom/django-components/actions/runs/4518425680/jobs/7958298706

Relevant parts of tox.ini

[tox]
envlist =
  py36-django{32}
  py37-django{32}
  py38-django{32,40}
  py39-django{32,40}
  py310-django{40}
  flake8
  isort

[gh-actions]
python =
  3.6: py36
  3.7: py37
  3.8: py38
  3.9: py39
  3.10: py310, flake8, isort
fail_on_no_env = True

Relevant parts of tests.yml

name: Run tests

on:
  - push
  - pull_request
  - workflow_dispatch

jobs:
  build:
    runs-on: ubuntu-20.04
    strategy:
      matrix:
        python-version: ['3.6', '3.7', '3.8', '3.9', '3.10']

    steps:
    - uses: actions/checkout@v3
    - name: Set up Python ${{ matrix.python-version }}
      uses: actions/setup-python@v4
      with:
        python-version: ${{ matrix.python-version }}
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        python -m pip install tox tox-gh-actions
    - name: Run tests
      run: tox -vv

TypeError comparing options.parallel to None

Describe the bug
I'm coming back to a project I haven't touched for 7 months.
It follows the examples fairly closely.
All the CI is broken with the following.

Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.10.10/x64/bin/tox", line 8, in <module>
    sys.exit(run())
  File "/opt/hostedtoolcache/Python/3.10.10/x64/lib/python3.10/site-packages/tox/run.py", line 1[9](https://github.com/tolomea/django-data-browser/actions/runs/4341383293/jobs/7580915003#step:6:10), in run
    result = main(sys.argv[1:] if args is None else args)
  File "/opt/hostedtoolcache/Python/3.[10](https://github.com/tolomea/django-data-browser/actions/runs/4341383293/jobs/7580915003#step:6:11).10/x64/lib/python3.10/site-packages/tox/run.py", line 41, in main
    result = provision(state)
  File "/opt/hostedtoolcache/Python/3.10.10/x64/lib/python3.10/site-packages/tox/provision.py", line 105, in provision
    MANAGER.tox_add_core_config(state.conf.core, state)
  File "/opt/hostedtoolcache/Python/3.10.10/x64/lib/python3.10/site-packages/tox/plugin/manager.py", line 66, in tox_add_core_config
    self.manager.hook.tox_add_core_config(core_conf=core_conf, state=state)
  File "/opt/hostedtoolcache/Python/3.10.10/x64/lib/python3.10/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/opt/hostedtoolcache/Python/3.10.10/x64/lib/python3.10/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/opt/hostedtoolcache/Python/3.10.10/x64/lib/python3.10/site-packages/pluggy/_callers.py", line 60, in _multicall
    return outcome.get_result()
  File "/opt/hostedtoolcache/Python/3.10.10/x64/lib/python3.10/site-packages/pluggy/_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/opt/hostedtoolcache/Python/3.10.10/x64/lib/python3.10/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/opt/hostedtoolcache/Python/3.10.10/x64/lib/python3.10/site-packages/tox_gh_actions/plugin.py", line 56, in tox_add_core_config
    if not is_log_grouping_enabled(config.options):
  File "/opt/hostedtoolcache/Python/3.10.10/x64/lib/python3.10/site-packages/tox_gh_actions/plugin.py", line [20](https://github.com/tolomea/django-data-browser/actions/runs/4341383293/jobs/7580915003#step:6:21)7, in is_log_grouping_enabled
    if hasattr(options, "parallel") and options.parallel > 0:
TypeError: '>' not supported between instances of 'NoneType' and 'int'

That particular one came off py3.10 with the following:

Successfully installed cachetools-5.3.0 chardet-5.1.0 colorama-0.4.6 distlib-0.3.6 filelock-3.9.0 packaging-23.0 platformdirs-3.1.0 pluggy-1.0.0 pyproject-api-1.5.0 tomli-2.0.1 tox-4.4.6 tox-gh-actions-3.0.0 virtualenv-20.20.0

Any pointers on what to check?

gh-actions running incorrect tox envs

Relevant parts of my tox.ini look like:

[tox]
envlist =
    py37-cov
    py37-astropy{40,lts}
    py37-devdeps
    py37-numpy{117,118,119}
    py38
    build_docs
    linkcheck

[gh-actions]
python =
    3.7: py37-cov, py37-numpy, py37-astropy, build_docs
    3.8: py38

And I followed the README here to set up my gh actions config file: https://github.com/adrn/pyia/blob/main/.github/workflows/tests.yml

But it looks like the wrong envs are running with versions of Python. For example, in the py3.7 ubuntu test, only py37-cov and build_docs are run. In py3.8 macos, all of the envs are run. https://github.com/adrn/pyia/runs/861918140

Do you have any tips?

Thanks!

compatibility with Conda

Question
Is tox-gh-actions compatible with tox-conda?

Context
I have a project where I use tox and tox-conda. I use tox-conda because the project has dependencies that exist only in conda. However. I could never make tox-gh-actions work properly. I tried last year and now. Currently, tox runs all environments despite tox-gh-actions being installed and configured in tox.ini. I provide the links below:

Additional context
The changes I am trying to apply are in joaomcteixeira/taurenmd#59

Migrate plugin under tox-dev

I am a (tox/pytest/sphinx) plugin author myself and I am worried about plugins relying on a single person. Too often I was hit by either abandonware or other problems related to the fact that the project relied on a single person, which lost interest, or become permanently unavailable (yep, we all have an unknown expiration date).

Few years ago I started moving my projects to github orgs aimed at maintaining plugins long term, such as @tox-dev @pytest-dev @sphinx-contrib @jazzband or @PyCQA. I made a personal rule: avoid starting to use any plugin that is not already doing that (to some extent I do the same with python libraries in general).

Please really consider doing that move, you will keep your full owner control over the repository and I really doubt anyone else would step-in to make changes unless you become unavailable.

Shortly, future proof your kid project ;)

Can't make it work

Hi there,
I am trying to migrate from Travis to github action and I want to use tox-gh-actions. I tried a lot of things but I can't make it work.
Here is the current state:

Depending of my tries, I get one of the following errors:

  • if I call tox -e ${{ matrix.python-version }}, I get: ERROR: unknown environment '2.7' (or 3.6 or whatever) so it seems that tox-gh-actions does nothing in this case.
  • if I just call tox, all environments listed in envlist are run, so I get a ERROR: py35-sqla11: InterpreterNotFound: python3.5 (or whatever version).

Do you see anything wrong in these files?

Thank you very much!

Great addtion to the tox ecosystem (tox-dev adoption)

While I did not used this action yet, I was contemplating the idea for a very long time, as most of my projects are pure python and already using tox. This action is supposed to save us a huge amount of time maintaining, the unfriendly, github actions pipelines.

As tox4 work is progressing and is introducing notable changes, I wonder if it would not be a good idea to host this project under the @tox-dev organization, alongside other tox related projects, including plugins and CI/CD helpers.

tox: command not found

Github Actions seems to be unwilling to bow to tox somehow, not sure what I'm doing wrong. Had the same problem before without using tox-gh-actions. The worflow will start, but when it comes to tox, the log reports:

tox: command not found

Currently my workflow looks as follow:


on:
    pull_request:
        branches: [ master ]

jobs:
    test_pull_request:
        runs-on: ubuntu-latest
        strategy:
            max-parallel: 2
            matrix:
                python-version: [ 3.7, 3.8 ]
        steps:
            - uses: actions/checkout@v1
            - name: Setup Python
              uses: actions/setup-python@v2
              with:
                python-version: ${{ matrix.python }}
            - name: Install Tox and any other packages
              run: |
                  python -m pip install --upgrade pip
                  pip install tox tox-gh-actions
            - name: Run tox
              run: tox

And my tox.ini:

envlist = py37, py38, flake8, mypy

[travis]
python =
    3.8: py38
    3.7: py37

[testenv:flake8]
description = Linter with flake8
basepython = python
deps = flake8
commands = flake8 simpleor tests

[testenv:mypy]
description = Static type check with mypy
basepython = python
deps = mypy
commands = mypy simpleor tests --ignore-missing-imports

[testenv]
setenv =
    PYTHONPATH = {toxinidir}
deps =
    -r {toxinidir}/requirements.txt
; If you want to make tox run the tests with the same versions, create a
; requirements.txt with the pinned versions and uncomment the following line:
;     -r{toxinidir}/requirements.txt
commands =
    pip install -U pip
    pytest --basetemp={envtmpdir}

[gh-actions]
python =
    3.7: py37
    3.8: py38

Running tox locally is working fine (MacOs Catalina 10.15.7, python 3.8.5, tox 3.20.1).

pypy2

Describe the bug
The docs here suggest using pypy-2.7: pypy2-.., however I believe it should be pypy-2.7: pypy-...

Using 'pypy2' results in

| Running tox with tox-gh-actions
| $ tox
[CI/test]   โ“  ::add-matcher::/opt/hostedtoolcache/PyPy/2.7.18/x64/site-packages/tox_gh_actions/matcher.json
| No handlers could be found for logger "filelock"
| GLOB sdist-make: /Users/jayvdb/django/django-rest-framework-braces/setup.py
| pypy2-django11-drflatest create: /Users/jayvdb/django/django-rest-framework-braces/.tox/pypy2-django11-drflatest
| ::group::tox: pypy2-django11-drflatest
| ERROR: InterpreterNotFound:
| ___________________________________ summary ____________________________________
| ERROR:  pypy2-django11-drflatest: InterpreterNotFound:
| ::remove-matcher owner=tox-gh-actions-warning-error::

AttributeError: 'Config' object has no attribute 'isolated_build_env'

Describe the bug

The plugin seems to be having trouble after the latest release?

To Reproduce

Expected behavior
That the plugin does not raise an exception when running on GitHub Actions

Additional context

Dependencies Installed
 backports.entry-points-selectable-1.1.0 distlib-0.3.3 filelock-3.0.12 packaging-21.0 platformdirs-2.3.0 pluggy-1.0.0 py-1.10.0 pyparsing-2.4.7 six-1.16.0 toml-0.10.2 tox-3.24.4 tox-gh-actions-2.8.0 virtualenv-20.8.0
Stack Trace
 Traceback (most recent call last):
.tox create: /home/runner/work/jira/jira/.tox/.tox
  File "/opt/hostedtoolcache/Python/3.8.12/x64/bin/tox", line 8, in <module>
    sys.exit(cmdline())
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/tox/session/__init__.py", line 44, in cmdline
    main(args)
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/tox/session/__init__.py", line 69, in main
    exit_code = session.runcommand()
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/tox/session/__init__.py", line 187, in runcommand
    return provision_tox(provision_tox_venv, self.config.args)
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/tox/session/commands/provision.py", line 10, in provision_tox
    ensure_meta_env_up_to_date(provision_venv)
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/tox/session/commands/provision.py", line 24, in ensure_meta_env_up_to_date
    if provision_venv.setupenv():
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/tox/venv.py", line 629, in setupenv
    status = self.update(action=action)
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/tox/venv.py", line 273, in update
    self.hook.tox_testenv_create(action=action, venv=self)
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pluggy/_callers.py", line 60, in _multicall
    return outcome.get_result()
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pluggy/_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/tox_gh_actions/plugin.py", line 66, in tox_testenv_create
    start_grouping_if_necessary(venv)
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/tox_gh_actions/plugin.py", line 105, in start_grouping_if_necessary
    if envname == envconfig.config.isolated_build_env:
AttributeError: 'Config' object has no attribute 'isolated_build_env'
Error: Process completed with exit code 1.

Strange results with pypy36 and pypy37

I want to test my project against both PyPy 3.6 and 3.7. My tox-gh-actions is set up in my tox.ini like this:

[gh-actions]
python =
    3.6: py36
    3.7: py37
    3.8: py38
    3.9: py39
    3.10: py310
    pypy-3.6: pypy36
    pypy-3.7: pypy37

Here's a run of my GitHub Actions workflow: https://github.com/brechtm/rinohtype/actions/runs/515676468. For the pypy-3.6 unit tests jobs on Ubuntu and macOS, tox-gh-actions produces an empty envlist. The pypy-3.7 job works correctly (['pypy37'] envlist).

On Windows, the PyPy jobs actually run on CPython 3.9. But I think that's not an issue with tox-gh-actions since the Poetry venv is created using CPython 3.9 for some reason.

While the pypy36 and pypy37 envs aren't explicitly listed in the tox docs, they do seem to do the right thing when I run tox locally. I have verified this with PyPy 3.7 on macOS. I don't have PyPy 3.6 installed, but at least tox is attempting to run the correct interpreter:

$ tox -e pypy36
pypy36 create: /Users/brechtm/Documents/Code/rinohtype/.tox/pypy36
ERROR: InterpreterNotFound: pypy3.6
_____________________________________________ summary ______________________________________________
ERROR:  pypy36: InterpreterNotFound: pypy3.6

Trouble getting tox to build anything from the matrix

Probably I just configured something wrong, but I'm having trouble getting tox to run any tests.

https://github.com/stefanfoulis/django-sendsms-gh-ci-experiment/pull/1/checks?check_run_id=222843429

action: https://github.com/stefanfoulis/django-sendsms-gh-ci-experiment/blob/ci/gh-actions-experiment/.github/workflows/ci.yml
tox.ini: https://github.com/stefanfoulis/django-sendsms-gh-ci-experiment/blob/ci/gh-actions-experiment/tox.ini

In the same repo I also have a setup with travis using tox-travis which works.

Any help would be greatly appreciated.

Works with setup.cfg?

I am trying to get this to work for one of my project where I am using setup.cfg for tox configuration instead of tox.ini.

My tox configuration looks like this:

[tox]
envlist = py35, py36, py37, py36-lint

[gh-actions]
python =
    3.5: py35
    3.6: py36, py36-lint
    3.7: py37

[testenv]
deps =
    pytest
commands =
    pytest tests

[testenv:py36-lint]
deps =
    black
    flake8
    flake8-black
    mypy
commands =
    black --check --diff setup.py minilp tests
    flake8 minilp tests
    mypy minilp tests

I have copy/pasted your workflow (simply removed python 3.5), but only the py36-lint environment is tested by tox, as if I ran tox without your tox-gh-actions...

Any workaround?

My project is at https://github.com/Holt59/minilp/tree/packaging.

Add Pyston support

Pyston is part of the python action but it is still installable. It'd be great if you could add a way to detect Pyston and use that.

I tried using using TOXENV to make my tests run using Pyston but it kept creating a Python 3.8 virtual environment rather than using my Pyston environment

      - name: Set up Pyston
        if: ${{ matrix.python-version == 'pyston' }}
        env:
          TOXENV: pyston,pyston-setproctitle,pyston-uvloop
        run: |
          git clone https://github.com/pyenv/pyenv.git ~/.pyenv
          ~/.pyenv/bin/pyenv install pyston-2.2
          ~/.pyenv/bin/pyenv local pyston-2.2
      - name: unittest
        run: tox -vv

This was the tox verbose output

tox-gh-actions config: {'python': {'3.7': ['py37', 'py37-setproctitle', 'py37-uvloop'], '3.8': ['py38', 'py38-setproctitle', 'py38-uvloop'], '3.9': ['py39', 'py39-setproctitle', 'py39-uvloop'], 'pypy-3.7': ['pypy3', 'pypy3-setproctitle', 'pypy3-uvloop'], 'pyston': ['pyston', 'pyston-setproctitle', 'pyston-uvloop']}, 'env': {}}

I had to resort to adding an if statement and calling tox -e env,list to make it work. I suspect this is due to the scoping of env and it only applying to the current step.

Non-listed environment not used even if configured in gh-actions block of tox.ini

I have a tox.ini that looks like this (a bit stripped down):

[tox]
envlist = py3

[testenv]
# Dependencies prefixed with "ci: " are only installed when running in CI,
# which calls effectively "tox -e py3-ci" (see the gh-actions section below).
deps =
    pytest
    ci: pytest-github-actions-annotate-failures

[gh-actions]
# Mapping between the Python version used in GitHub Actions matrix builds, and
# the used Tox environment.
python =
    3.5: py3-ci
    3.6: py3-ci
    3.7: py3-ci
    3.8: py3-ci
    3.9: py3-ci

What I want to achieve is the following: run the py3-ci environment when running tox in GitHub Actions, but run the py3 environment otherwise. If I'd list the py3-ci environment in envlist, normal developers running tox would also get this environment if they don't override it on the command line.

Manually running tox py3-ci works fine and tox picks the right factor/environment, but when I run tox (without any argument) in GitHub actions, tox doesn't run anything.

Is there a bug somewhere that tries to match the environments in in the gh-actions key against the ones in envlist?

Enable log line grouping for more logs

It's nice to group log lines such as "installdeps" other than "run-test" but this is a bit tricky due to some technical difficulties. Things need to consider:


Current Behavior

$ tox -e flake8
GLOB sdist-make: /Users/yusuke/Development/github.com/ymyzk/tox-gh-actions/tox-gh-actions-test/setup.py
flake8 create: /Users/yusuke/Development/github.com/ymyzk/tox-gh-actions/tox-gh-actions-test/.tox/flake8
flake8 installdeps: flake8
flake8 inst: /Users/yusuke/Development/github.com/ymyzk/tox-gh-actions/tox-gh-actions-test/.tox/.tmp/package/1/UNKNOWN-0.0.0.zip
flake8 installed: flake8==3.9.2,mccabe==0.6.1,pycodestyle==2.7.0,pyflakes==2.3.1,UNKNOWN @ file:///Users/yusuke/Development/github.com/ymyzk/tox-gh-actions/tox-gh-actions-test/.tox/.tmp/package/1/UNKNOWN-0.0.0.zip
::group::tox: flake8
flake8 run-test-pre: PYTHONHASHSEED='3474553861'
flake8 run-test: commands[0] | flake8 setup.py
::endgroup::
________________________________________________________________________________________ summary ________________________________________________________________________________________
  flake8: commands succeeded
  congratulations :)
$ tox -e flake8                                                                                                                                                        -? (git)-[master]
GLOB sdist-make: /Users/yusuke/Development/github.com/ymyzk/tox-gh-actions/tox-gh-actions-test/setup.py
flake8 inst-nodeps: /Users/yusuke/Development/github.com/ymyzk/tox-gh-actions/tox-gh-actions-test/.tox/.tmp/package/1/UNKNOWN-0.0.0.zip
flake8 installed: flake8==3.9.2,mccabe==0.6.1,pycodestyle==2.7.0,pyflakes==2.3.1,UNKNOWN @ file:///Users/yusuke/Development/github.com/ymyzk/tox-gh-actions/tox-gh-actions-test/.tox/.tmp/package/1/UNKNOWN-0.0.0.zip
::group::tox: flake8
flake8 run-test-pre: PYTHONHASHSEED='4256965523'
flake8 run-test: commands[0] | flake8 setup.py
::endgroup::
________________________________________________________________________________________ summary ________________________________________________________________________________________
  flake8: commands succeeded
  congratulations :)

Expected Behavior

$ tox -e flake8
GLOB sdist-make: /Users/yusuke/Development/github.com/ymyzk/tox-gh-actions/tox-gh-actions-test/setup.py
::group::tox: flake8
flake8 create: /Users/yusuke/Development/github.com/ymyzk/tox-gh-actions/tox-gh-actions-test/.tox/flake8
flake8 installdeps: flake8
flake8 inst: /Users/yusuke/Development/github.com/ymyzk/tox-gh-actions/tox-gh-actions-test/.tox/.tmp/package/1/UNKNOWN-0.0.0.zip
flake8 installed: flake8==3.9.2,mccabe==0.6.1,pycodestyle==2.7.0,pyflakes==2.3.1,UNKNOWN @ file:///Users/yusuke/Development/github.com/ymyzk/tox-gh-actions/tox-gh-actions-test/.tox/.tmp/package/1/UNKNOWN-0.0.0.zip
flake8 run-test-pre: PYTHONHASHSEED='3474553861'
flake8 run-test: commands[0] | flake8 setup.py
::endgroup::
________________________________________________________________________________________ summary ________________________________________________________________________________________
  flake8: commands succeeded
  congratulations :)
$ tox -e flake8                                                                                                                                                        -? (git)-[master]
GLOB sdist-make: /Users/yusuke/Development/github.com/ymyzk/tox-gh-actions/tox-gh-actions-test/setup.py
::group::tox: flake8
flake8 inst-nodeps: /Users/yusuke/Development/github.com/ymyzk/tox-gh-actions/tox-gh-actions-test/.tox/.tmp/package/1/UNKNOWN-0.0.0.zip
flake8 installed: flake8==3.9.2,mccabe==0.6.1,pycodestyle==2.7.0,pyflakes==2.3.1,UNKNOWN @ file:///Users/yusuke/Development/github.com/ymyzk/tox-gh-actions/tox-gh-actions-test/.tox/.tmp/package/1/UNKNOWN-0.0.0.zip
flake8 run-test-pre: PYTHONHASHSEED='4256965523'
flake8 run-test: commands[0] | flake8 setup.py
::endgroup::
________________________________________________________________________________________ summary ________________________________________________________________________________________
  flake8: commands succeeded
  congratulations :)

tox-gh-actions prevents an explicitly specified TOXENV from being run

When explicitly specifying a tox environment to run by setting the TOXENV environment variable, tox-gh-actions will not run any tox environment. It doesn't matter whether the environment is one that is specified in tox.ini or not.

$ GITHUB_ACTION=1 TOXENV=regression tox -v
using tox.ini: /Users/brechtm/Documents/Code/rinohtype/tox.ini (pid 13403)
using tox-3.20.0 from /Users/brechtm/Documents/Code/rinohtype/.venv/lib/python3.8/site-packages/tox/__init__.py (pid 13403)
_________________________________________ summary _________________________________________
  congratulations :)

`tox-gh-actions` selects too many environments

I have configured tox so that I can run tox -e unit for unit tests and tox -e unit-cover for unit tests with coverage. It is impossible for me to use tox-gh-actions to run only tox -e unit if both unit and unit-cover are listed in envlist (because there are some GHA workflows in which I need coverage, and others in which I do not need it). If I try to filter by factors, tox-gh-actions selects both environments and runs them sequentially. Is there a solution other than not using tox-gh-actions?

Can't run envs that aren't in envlist?

I'm currently trying to set this up for a repo, but I'm having a bit of trouble with one specific thing.

It seems this can't handle environments that aren't in envlist?

The idea here is that I want to have -ci environments which aren't used if you just run tox normally. They're only used when explicitly specified. That's why I don't want to put them in envlist: whatever you put in envlist is what will be run if you just run tox with no -e argument, and there's no way to change this, so far as I can tell.

The config I have:

[tox]
envlist = py{27,36,37,38,39}

[gh-actions]
python =
    2.7: py27
    3.6: py36-ci
    3.7: py37-ci
    3.8: py38-ci
    3.9: py39-ci

[testenv]
deps =
    -r{toxinidir}/install.requires
    -r{toxinidir}/tests.requires
    ci: -r{toxinidir}/ci.requires

commands =
    py.test
    ci: py.test --cov-report term-missing --cov-report xml --cov openqa_client
    ci: diff-cover coverage.xml --fail-under=90
    ci: diff-quality --violations=pylint --fail-under=90
setenv =
    PYTHONPATH = {toxinidir}

works fine when I test from the command line. If I just run tox, the py27, py36 etc. environments - which don't run the CI commands - are run. If I run tox -epy{27,36,37,38,39}-ci, tox runs the CI environments and commands.

But if you look at the test results in the pull request, this doesn't seem to be working there. In tox.ini I configured all the Python 3 environments as I actually want them to be set up:

3.6: py36-ci
3.7: py37-ci
3.8: py38-ci
3.9: py39-ci

but it seems like when tox-gh-actions goes to run these, it just sort of does nothing:

2020-02-28T01:10:13.0230470Z ##[group]Run tox
2020-02-28T01:10:13.0230678Z ๏ฟฝ[36;1mtox๏ฟฝ[0m
2020-02-28T01:10:13.0267544Z shell: /bin/bash -e {0}
2020-02-28T01:10:13.0267665Z env:
2020-02-28T01:10:13.0267776Z   pythonLocation: /opt/hostedtoolcache/Python/3.6.10/x64
2020-02-28T01:10:13.0267870Z ##[endgroup]
2020-02-28T01:10:13.2767035Z ___________________________________ summary ____________________________________
2020-02-28T01:10:13.2769413Z   congratulations :)

I set the Python 2.7 line to use the non-CI environment, as a check:

2.7: py27

and just that environment works correctly:

2020-02-28T01:10:13.9845791Z ##[group]Run tox
2020-02-28T01:10:13.9845964Z ๏ฟฝ[36;1mtox๏ฟฝ[0m
2020-02-28T01:10:13.9877602Z shell: /bin/bash -e {0}
2020-02-28T01:10:13.9877667Z env:
2020-02-28T01:10:13.9877745Z   pythonLocation: /opt/hostedtoolcache/Python/2.7.17/x64
2020-02-28T01:10:13.9877826Z ##[endgroup]
2020-02-28T01:10:14.1993163Z py27 create: /home/runner/work/openQA-python-client/openQA-python-client/.tox/py27
2020-02-28T01:10:14.6157194Z py27 installdeps: -r/home/runner/work/openQA-python-client/openQA-python-client/install.requires, -r/home/runner/work/openQA-python-client/openQA-python-client/tests.requires
2020-02-28T01:10:19.7790850Z py27 installed: DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support,atomicwrites==1.3.0,attrs==19.3.0,certifi==2019.11.28,chardet==3.0.4,configparser==4.0.2,contextlib2==0.6.0.post1,freezegun==0.3.15,funcsigs==1.0.2,idna==2.9,importlib-metadata==1.5.0,mock==3.0.5,more-itertools==5.0.0,packaging==20.1,pathlib2==2.3.5,pluggy==0.13.1,py==1.8.1,pyparsing==2.4.6,pytest==4.6.9,python-dateutil==2.8.1,requests==2.23.0,scandir==1.10.0,six==1.14.0,urllib3==1.25.8,wcwidth==0.1.8,zipp==1.2.0
2020-02-28T01:10:19.7795701Z py27 run-test-pre: PYTHONHASHSEED='3292039813'
2020-02-28T01:10:19.7796265Z py27 run-test: commands[0] | py.test
2020-02-28T01:10:20.2531722Z ============================= test session starts ==============================
2020-02-28T01:10:20.2532468Z platform linux2 -- Python 2.7.17, pytest-4.6.9, py-1.8.1, pluggy-0.13.1
2020-02-28T01:10:20.2533020Z cachedir: .tox/py27/.pytest_cache
2020-02-28T01:10:20.2533388Z rootdir: /home/runner/work/openQA-python-client/openQA-python-client
2020-02-28T01:10:20.4473469Z collected 15 items
2020-02-28T01:10:20.4473548Z 
2020-02-28T01:10:20.8115094Z tests/test_client.py ...............                                     [100%]
2020-02-28T01:10:20.8115567Z 
2020-02-28T01:10:20.8115679Z ========================== 15 passed in 0.56 seconds ===========================
2020-02-28T01:10:20.8318523Z ___________________________________ summary ____________________________________
2020-02-28T01:10:20.8318657Z   py27: commands succeeded
2020-02-28T01:10:20.8318744Z   congratulations :)
2020-02-28T01:10:20.8447583Z Cleaning up orphan processes

which is what makes me think the problem is running an environment that's not in envlist...

Error in alpine Linux with matchers

When using the last version of tox-gh-actions on alpine Linux it fails with this error:

Error: Unable to process command '::add-matcher::/usr/local/lib/python3.10/site-packages/tox_gh_actions/matcher.json' successfully. 

Error: Could not find a part of the path '/usr/local/lib/python3.10/site-packages/tox_gh_actions/matcher.json'. 

original envconfigs: ['.package', 'py310-cov', 'py311', 'py310', 'py39', 'py38', 'lint']

Python 3.11 Support

Describe the bug
This does currently not support running Python 3.11 environments.

To Reproduce
Add an 3.11: py311 environment to the [gh-actions] sections and run the tests from within a Github action with Python 3.11.

Expected behavior
I expect the tests to run.

Additional context
Tox immediately exists, printing Warning: WARNING: tox-gh-actions couldn't find environments matching the provided factors from envlist. Please use tox -vv to get more detailed logs..

Package Installation failing on github actions.

Describe the bug
When running tox on github actions i'm facing an OSError: Could not find a suitable TLS CA certificate bundle, invalid path: /home/runner/work/cash-opera-backend/cash-opera-backend/.tox/py/lib/python3.9/site-packages/certifi/cacert.pem
Some packages get installed successfully, but the majority fails, here's a traceback of a package installation failure (This gets repeated for all the packages that failed to install).

  ERRORS:
  update certifi failed:
  Traceback (most recent call last):
    File 
  "/opt/hostedtoolcache/Python/3.9.13/x64/lib/python3.9/concurrent/futures/thread.
  py", line 58, in run
      result = self.fn(*self.args, **self.kwargs)
    File 
  "/home/runner/work/cash-opera-backend/cash-opera-backend/.tox/py/lib/python3.9/s
  ite-packages/pdm/installers/synchronizers.py", line 265, in update_candidate
      self.manager.install(can)
    File 
  "/home/runner/work/cash-opera-backend/cash-opera-backend/.tox/py/lib/python3.9/s
  ite-packages/pdm/installers/manager.py", line 39, in install
      installer(str(prepared.build()), self.environment, prepared.direct_url())
    File 
  "/home/runner/work/cash-opera-backend/cash-opera-backend/.tox/py/lib/python3.9/s
  ite-packages/pdm/models/candidates.py", line 359, in build
      self.obtain(allow_all=False)
    File 
  "/home/runner/work/cash-opera-backend/cash-opera-backend/.tox/py/lib/python3.9/s
  ite-packages/pdm/models/candidates.py", line 428, in obtain
      result = finder.download_and_unpack(
    File 
  "/home/runner/work/cash-opera-backend/cash-opera-backend/.tox/py/lib/python3.9/s
  ite-packages/unearth/finder.py", line 356, in download_and_unpack
      file = unpack_link(
    File 
  "/home/runner/work/cash-opera-backend/cash-opera-backend/.tox/py/lib/python3.9/s
  ite-packages/unearth/preparer.py", line 300, in unpack_link
      resp = session.get(link.normalized, stream=True)
    File 
  "/home/runner/work/cash-opera-backend/cash-opera-backend/.tox/py/lib/python3.9/s
  ite-packages/requests/sessions.py", line 600, in get
      return self.request("GET", url, **kwargs)
    File 
  "/home/runner/work/cash-opera-backend/cash-opera-backend/.tox/py/lib/python3.9/s
  ite-packages/requests/sessions.py", line 587, in request
      resp = self.send(prep, **send_kwargs)
    File 
  "/home/runner/work/cash-opera-backend/cash-opera-backend/.tox/py/lib/python3.9/s
  ite-packages/requests/sessions.py", line 701, in send
      r = adapter.send(request, **kwargs)
    File 
  "/home/runner/work/cash-opera-backend/cash-opera-backend/.tox/py/lib/python3.9/s
  ite-packages/cachecontrol/adapter.py", line 57, in send
      resp = super(CacheControlAdapter, self).send(request, **kw)
    File 
  "/home/runner/work/cash-opera-backend/cash-opera-backend/.tox/py/lib/python3.9/s
  ite-packages/requests/adapters.py", line 460, in send
      self.cert_verify(conn, request.url, verify, cert)
    File 
  "/home/runner/work/cash-opera-backend/cash-opera-backend/.tox/py/lib/python3.9/s
  ite-packages/requests/adapters.py", line 263, in cert_verify
      raise OSError(
  OSError: Could not find a suitable TLS CA certificate bundle, invalid path: 
  /home/runner/work/cash-opera-backend/cash-opera-backend/.tox/py/lib/python3.9/si
  te-packages/certifi/cacert.pem

To Reproduce

  • GitHub Actions configuration
name: Tests

on:
  - push
  - pull_request

jobs:
  test:
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: [ubuntu-20.04]
        python-version: ["3.9.13"]

    steps:
      - uses: actions/checkout@v2
      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v2
        with:
          python-version: ${{ matrix.python-version }}
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip setuptools wheel
          pip install tox tox-gh-actions certifi
      - name: Test with tox
        run: tox

  build:
    runs-on: ubuntu-20.04
    steps:
      - uses: actions/checkout@v2
      - name: Set up Docker Buildx
        id: buildx
        uses: docker/setup-buildx-action@v1
      - name: Build Image
        id: docker_build
        uses: docker/build-push-action@v2
        with:
          context: ./
          file: ./Dockerfile
          push: false
  • tox and tox-gh-actions configuration
[tox]
minversion = 3.25.1
envlist = py39
isolated_build = true

[gh-actions]
python =
    3.9.13:   py39

[testenv]
setenv =
    PYTHONPATH = {toxinidir}/src/
    DJANGO_SETTINGS_MODULE = app.settings.test
    DJANGO_SECRET_KEY = ABCdefGHIjklMNOpqrSTUvwxYZ123456789$%*
    DJANGO_DEBUG=True
    BROKER_URL='Empty'
    BROKER_TRANSPORT='redis'
    PDM_IGNORE_SAVED_PYTHON="1"
    REDIS_HOST=redis
    REDIS_PORT=6379
    ACCESS_TOKEN_LIFETIME_MINUTES=15
    REFRESH_TOKEN_LIFETIME_DAYS=1
deps =
    pdm
    setuptools
    wheel

commands =
    pdm install --dev
    pdm run python src/manage.py collectstatic --noinput
    pdm run pytest --basetemp={envtmpdir}

    pdm run black src --check
    pdm run mypy src
  • pyproject.toml
[tool.mypy]
mypy_path = "src"
check_untyped_defs = true
disallow_any_generics = true
ignore_missing_imports = true
no_implicit_optional = true
show_error_codes = true
strict_equality = true
warn_redundant_casts = true
warn_return_any = true
warn_unreachable = true
warn_unused_configs = true
no_implicit_reexport = true
allow_redefinition = true

[tool.black]
line-length = 100
extend-exclude = """
migrations
"""

[tool.isort]
profile = "black"

[tool.pdm]
[[tool.pdm.source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[tool.pdm.scripts]
_.env_file = ".env.dev"
collectstatic.cmd = "python manage.py collectstatic --noinput"
makemigrations.cmd = "python manage.py makemigrations"
migrate.cmd = "python manage.py migrate"
server.cmd = "python manage.py runserver"
createsuperuser.cmd = "python manage.py createsuperuser"
makemessages.shell = "cd src && pdm run python manage.py makemessages"
compilemessages.shell = "cd src && pdm run python manage.py compilemessages"
tests = { env_file = ".env.test", shell = "rm -r test_media || true && mkdir test_media && coverage run --source=src/ --rcfile=.coveragerc -m pytest tests/ && coverage html && google-chrome htmlcov/index.html"}

[tool.pdm.dev-dependencies]
dev = [
    "black==22.3.0",
    "pytest==7.0.1",
    "pytest-django>=4.5.2",
    "mypy==0.961",
    "django-debug-toolbar>=3.5",
    "coverage==6.3.2",
    "pre-commit>=2.19.0",
    "isort>=5.10.1",
    "pytest-socket>=0.5.1",
    "model-bakery>=1.9.0",
]
[build-system]
requires = ["pdm-pep517>=1.0.0"]
build-backend = "pdm.pep517.api"

[tool.pytest.ini_options]
testpaths = [
    "tests/",
]
addopts = "--rootdir ./tests --disable-socket"

[project]
name = "cashopera"
version = "0.1.0"
description = "Cash Opera Django App"
authors = [
    {name = "Leandro de Souza", email = "[email protected]"},
]
dependencies = [
    "Django==4.1",
    "djangorestframework>=3.13.1",
    "celery[redis]>=5.2.7",
    "whitenoise>=6.2.0",
    "django-environ>=0.9.0",
    "django-jazzmin>=2.5.0",
    "drf-spectacular[sidecar]>=0.22.1",
    "django-cors-headers>=3.13.0",
    "psycopg2-binary>=2.9.3",
    "setuptools>=62.6.0",
    "channels>=3.0.5",
    "channels-redis>=3.4.0",
    "daphne>=3.0.2",
    "sendgrid>=6.9.7",
    "django-storages>=1.12.3",
    "Pillow>=9.2.0",
    "flower>=1.1.0",
    "validate-docbr>=1.10.0",
    "phonenumbers>=8.12.54",
    "boto3>=1.24.76",
    "requests>=2.28.1",
    "beautifulsoup4>=4.11.1",
    "django-filter>=22.1",
]
requires-python = ">=3.9.13"
license = {text = "MIT"}

[project.optional-dependencies]

Since the repository i'm working on is not public, here's the downloaded log file from the failed action.
logs_74.zip

Expected behavior
I was not facing this issue since last week, all the packages were being installed and the action were ran correctly.

Additional context
I'm using pdm 2.1 as a project/dependency manager.
Running tox locally doesn't give me the same error.

I had take a look in some in this issue on my way here: tox-dev/tox - tox fails due to ssl #809

Can the warning about "no found environment" be turned into an error?

Question
When no environment is configured, tox-gh-actions gracefully degrades to a no-op and prints a warning. Is there a way to turn this warning into an error?

Context
pytest-asyncio uses tox-gh-actions. We run the tests against different CPython versions and forgot to update the 3.11-dev CI environment with 3.11. Nobody noticed for quite a few weeks that the tests for Python 3.11 were not actually run.

I would prefer to get an error that results in a failing CI pipeline. Is there some configuration to make this possible?

No coverage report data on windows-latest workflows

Describe the bug
Since the release of tox-gh-actions 3.0.0 the qpsolvers project has been experiencing Windows CI issues qpsolvers/qpsolvers#141. The symptom is:

py37-windows: commands[7]> coverage report --include=\"qpsolvers/*\"
  No data to report.

Which makes the CI fail. Version-pinning tox-gh-actions to 2.12.0 removes the symptom, as done e.g. in this job.

To Reproduce

  • GitHub Actions configuration: test.yml
  • tox and tox-gh-actions configuration: tox.ini
  • Link to the GitHub Actions job if a repository is public: for instance this one

Expected behavior

Successful coverage report generation.

Additional context

  • The issue only happens only on windows-latest images (ubuntu-latest and macos-latest images work fine).
  • The issue started happening two days ago with this job, which is a minor change following a successful build.
  • Previously this job worked fine. It installed tox-4.0.16 and tox_gh_actions-2.12.0.
  • Version-pinning tox-gh-actions to 2.12.0 removes the symptom, as done e.g. in this job. It installs tox-3.28.0 and tox_gh_actions-2.12.0.

codecov not working

Describe the bug
I switched from Travis CI to GitHub actions for testing my django packages.
I'm using the same tox file, but now the coverage report is not uploaded to codecov anymore, as you can see here there are no more reports uploaded since September 2021.

To Reproduce

Expected behavior
The coverage report should be uploaded to codecov.

Additional context
By running tox locally, actually the coverage is 92%.

can it work with macos?

I got this error:
/Users/runner/work/_temp/b38d46ba-4cfc-48f7-80a9-1be0929de1f0.sh: line 1: tox: command not found

Valid versioning syntax when specifying Python version in tox.ini

Question
Hi! Does tox-gh-actions support x.y-dev syntax when specifying the Python version in tox.ini?

Context
For context, this is a downstream issue in PyCQA/flake8-bugbear, where in our config we're able to specify a release candidate version of 3.11 (using the syntax detailed here), for GH Actions and install it, but this isn't being picked up by tox as matching any environments from the config envlist.

For an example of the failure, see this CI run. Our config is here.

Additional context
I've been able to run this by replacing 3.11.0-rc - 3.11: py311 with 3.11: py311, but in the future we may want to support Python versions before full release. Looking at the source code here, I'm thinking that tox-gh-actions is comparing the Python version requested by the config with the major.minor-formatted version of the actual distribution. To support this, it might be necessary to take parse something like major.minor.micro-releaselevel (as supplied by sys.versioninfo).

tox v4 support

tox v4 is work in progress at https://github.com/tox-dev/tox/tree/rewrite. It says

Compatibility wise we aim to be (excluding some weird edge cases) configuration file compatible with tox 3. We'll not be API compatible though (all plugins will break).

so we'll likely need to make some changes. It seems the new API for plugins is not well documented at this point. We can start working on this later.


Plan for tox v4 support:

  • Preparation
    • Drop Python 2.7 and 3.5 support
    • Remove deprecated features: old-style PyPy key
    • Use annotations for type hints
    • Remove pytest v5
  • tox v4 support
    • Initial support
    • Improve test coverage
    • Update documentation
    • Support log line grouping
  • Release
    • Release tox-gh-actions v3.0.0a1 which is tested with tox v4.0.0a9
    • Release tox-gh-actions v3.0.0 once tox v4.0.0 is released

Migrate to tox-dev and v4 support

Hello, would you consider moving the project under the tox-dev umbrella? See documentation under https://tox.readthedocs.io/en/rewrite/plugins.html#adoption-of-a-plugin-under-tox-dev-github-organization

Furthermore, tox v4 is getting ready and we'd like to make sure this plugin is supported from day 1, we're collecting feature gaps for this under tox-dev/tox#1974. Would be great if you could join our development chat under https://discord.gg/tox so we can assist with this. If you do so please drop in a line in the #plugin chat with the name of the repository you maintain. Thanks!

fail when envlist is empty

It took me a while to realize I had the same issue as #44.

I think the run should fail if the resulting envlist is empty, that way one doesn't accidentally think the action succeeded when nothing was actually done.

[regression] tox-gh-actions >= 3 breaks tox >= 4

OVERVIEW

The tox-gh-actions somehow cause dependencies to not be installed when using recent tox.

The following output is available when running the build manually not in github, notice that the pre-commit which is at the dev dependency is not installed.

$ tox
.tox create: xxx/test_project/.tox/.tox
.tox installdeps: tox-gh-actions==3.*, tox >= 3.21.4
ROOT: tox-gh-actions won't override envlist because tox is not running in GitHub Actions
.pkg: install_requires> python -I -m pip install setuptools wheel
.pkg: _optional_hooks> python xxx/test_project/.tox/.tox/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg: get_requires_for_build_sdist> python xxx/test_project/.tox/.tox/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg: install_requires_for_build_sdist> python -I -m pip install setuptools
.pkg: build_sdist> python xxx/test_project/.tox/.tox/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta
test: install_package> python -I -m pip install --force-reinstall --no-deps xxx/test_project/.tox/.tmp/package/1/test-project-0.0.0.tar.gz
test: commands[0]> pre-commit run --all-files
test: exit 2 (0.07 seconds) xxx/test_project> pre-commit run --all-files
.pkg: _exit> python xxx/test_project/.tox/.tox/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta
  test: FAIL code 2 (6.64=setup[6.57]+cmd[0.07] seconds)
  evaluation failed :( (6.67 seconds)

REPRODUCTION

Files

tox.ini

[tox]
isolated_build = True
envlist = test
requires =
#    tox==3.*
#    tox-gh-actions==2.*
    tox-gh-actions==3.*

[testenv]
extras = dev
commands =
    pre-commit run --all-files {posargs}

setup.cfg

[metadata]
name = test_project
long_description = file: README.md
long_description_content_type = text/markdown

[options]
packages = find_namespace:
setup_requires =
    setuptools

[options.extras_require]
dev =
    pre-commit

[egg_info]
tag_date = 0

pyproject.toml

[build-system]
requires = [
    "setuptools",
    "wheel",
]
build-backend = "setuptools.build_meta"

[project]
name = "test-project"
requires-python = ">=3.10"
version = "0.0.0"
dynamic = [
    "readme",
]

[[project.authors]]
name = "Test"
email = "[email protected]"

.pre-commit-config.yaml

repos:
  - repo: https://github.com/psf/black
    rev: 22.8.0
    hooks:
      - id: black

test_project/init.py

Invocation

  • When tox > 4 and tox-gh-actions==3.* fails.
  • When tox > 4 and tox-gh-action==2.* succeeds.
  • when tox==3.* and tox-gh-actions==2.* success.

Please play with the dependencies at tox.ini:

requires =
#    tox==3.*
#    tox-gh-actions==2.*
    tox-gh-actions==3.*

SUSPECTED CAUSE

The tox-gh-actions causes requirements of environment to be skipped when tox >= 4

EXPECTED BEHAVIOR

  • tox-gh-actions should not affect build environment when running out of github
  • tox-gh-actions should not fail with recent tox

Cannot find matcher.json after update to 2.9.0

Describe the bug
After the release of tox-gh-actions 2.9.0 our Github Actions CI is failing with an error that matcher.json cannot be found:

image

To Reproduce

  • GitHub Actions configuration
test:
    name: Unit test on supported platforms
    runs-on: ubuntu-latest
    container: python:${{ matrix.python-version }}
    strategy:
      matrix:
        python-version: ['3.6', '3.7', '3.8', '3.9', '3.10']
    steps:
    - uses: actions/checkout@v1

    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install tox tox-gh-actions
    - name: Test with tox
      run: tox
  • tox and tox-gh-actions configuration
[tox]
envlist = py36,py37,py38,py39,py310
isolated_build = True
[gh-actions]
python =
    3.6: py36
    3.7: py37
    3.8: py38
    3.9: py39
    3.10: py310
[testenv]
deps =
   ...
commands = pytest ./tests

Expected behavior
I expect the CI run to complete without error

Improve error reporting when a config is malformed

Problem
When a given configuration is malformed (: must be used instead of =) like the following example (#174)

python =
    3.7 = py37
    3.8 = py38
...

tox-gh-actions reports user-unfriendly error like

return {k: StrConvert.to_env_list(v).envs for k, v in parse_dict(value).items()}
File "/opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/site-packages/tox_gh_actions/plugin.py", line 264, in parse_dict
return dict((k.strip(), v.strip()) for k, v in pairs)
File "/opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/site-packages/tox_gh_actions/plugin.py", line 264, in <genexpr>
return dict((k.strip(), v.strip()) for k, v in pairs)
ValueError: not enough values to unpack (expected 2, got 1)
Error: Process completed with exit code 1.

Idea
It's better to detect malformed configurations and show user-friendly error messages instead.

Failed to import the site module in Windows Environment

Hi,

Not entirely sure if this is directly related to tox-gh-actions or not, but I seem to be having an issue with tox in Windows environments only.

In both macOS-10.14 and ubuntu-18.04 my action passes as expected, but in windows-2016 I am met with:

 Fatal Python error: initsite: Failed to import the site module
Traceback (most recent call last):
  File "D:\a\micropy-cli\micropy-cli\.tox\py37\lib\site.py", line 769, in <module>
    main()
  File "D:\a\micropy-cli\micropy-cli\.tox\py37\lib\site.py", line 746, in main
    paths_in_sys = addsitepackages(paths_in_sys)
  File "D:\a\micropy-cli\micropy-cli\.tox\py37\lib\site.py", line 279, in addsitepackages
    addsitedir(sitedir, known_paths)
  File "D:\a\micropy-cli\micropy-cli\.tox\py37\lib\site.py", line 202, in addsitedir
    addpackage(sitedir, name, known_paths)
  File "D:\a\micropy-cli\micropy-cli\.tox\py37\lib\site.py", line 170, in addpackage
    exec(line)
  File "<string>", line 1, in <module>
  File "D:\a\micropy-cli\micropy-cli\.tox\py37\lib\site-packages\win32\lib\pywin32_bootstrap.py", line 14, in <module>
    for maybe in site.getsitepackages():
AttributeError: module 'site' has no attribute 'getsitepackages'

Followed by:
ERROR: py37: InvocationError for command 'D:\a\micropy-cli\micropy-cli\.tox\py37\Scripts\python.EXE' -m pip freeze (exited with code 1)

The full checksuite can be viewed here
The full workflow can be found here
tox.ini file can be found here.

For now I will be trying a different windows environment to test if the issue persists. Thanks.

Grouping and extra output makes hard to find the error line

While trying to evaluate use of this plugin for one project where I have a chain of 4 tox environments run one after another, I discovered two problems with it, which makes it a blocker for adoption.

  • Undesired extra output is present that is not included in collapsed groups, mainly the 4 lines around recreate/installdeps/develop-inst/installed. These take more like 12-15 lines of console output due to the longer list of dependencies
  • The errored environment is already collapsed and user needs to find its line and expand it. That is bit of the finding the needle in the hay-stack (console), which requires scrolling few screens.

I suppose that some of these are part of the grouping limitation on GHA, but even so we should make a feature request to GHA to allow groups to determine their default state when the group is ended (that is the moment when you know if there was a failure or not on that group).

Still, including the extra lines into the group is clearly under tox/plugin responsibility and we should ensure that messages related to specific group are not logged outside the group.

Please note the extra lines on screenshot below, see the scrollbar size, user is expected to scroll ~2 screens up to find the broken group (which is not even highlighted), expand it and scroll does several more screens in order to find the real test failure. That is a clear regression from the case where we manually had on GHA step for each tox environment, as the failed steps are already marked as red and they contain all the output.

I hope this report will be useful for improving this action in the future.

Arbitrary Environment Factor Support?

First off, I'd like to express my appreciation and support of this project! It's a fairly interesting tool which performs well for my application. Kudos to the dev team!

I'm writing to see if it's possible to select runs for some complex generative envlist based on more factors than just "python"?

For example:

#tox.ini

[tox]
envlist =
    clean,
    docs,
    py{36,37}-cov{yes,no}-ml{yes,no},
    report
#.github/workflows/toxtest.yml

jobs:
  build:
    runs-on: ${{ matrix.platform }}
    strategy:
      fail-fast: false
      matrix:
        platform:
          - ubuntu-latest
          - macos-latest
          - windows-latest
        python-version: [3.6, 3.7]

...

I can only use tox-gh-actions to select environmens based on the python version in the build matrix. (by including the following in my tox.ini file

#tox.ini

[gh-actions]
python =
    3.6: py36-mlyes,report
    3.7: py37-mlno,report

Which runs py36-covyes-mlyes, py36-covno-mlyes, report and py37-covyes-mlyes, py37-covno-mlyes, report for python 3.6 and 3.7, respectively.

I would like to be able to include additional factor specifiers in the workflow job matrix which map to a reduced set of toxenvs from the full list.

Something like

#.github/workflows/toxtest.yml

jobs:
  build:
    runs-on: ${{ matrix.platform }}
    strategy:
      fail-fast: false
      matrix:
        platform:
          - ubuntu-latest
          - macos-latest
          - windows-latest
        python-version: [3.6, 3.7]
        coverage: [yes, no]  # new matrix parameter

...
#tox.ini

[gh-actions]
python,coverage =
    3.6,yes: py36-covyes,report
    3.6,no: py36-covno
    3.7,yes: py37-covyes,report
    3.7,no: py37-covno

This doesn't seem possible in the current version, but I may be mistaken?

Any tips on how to support such a workflow?

Update README Examples for Python 3.10

Python 3.10 requires quotes around "3.10" in YAML:

jobs:
  tox:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: [ "3.8", "3.9", "3.10" ]

I would recommend for consistency all of the python-version specifiers be updated with quotes, as above. This will reduce surprises for users down the road.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.