asottile / setup-py-upgrade Goto Github PK

View Code? Open in Web Editor NEW

147.0 5.0 7.0 204 KB

upgrade a setup.py to declarative metadata

License: MIT License

Python 100.00%

packaging setuptools

setup-py-upgrade's Issues

strips shebang lines

It looks like setup-py-upgrade strips leading comments for python files:

# Some comment

setup(name="blala")

results in the comment being stripped.

Consider adding support for `extras_require`

This project is kinda cool. Unfortunately, the current version doesn't support the extras_require section. Apparently, the most extreme case is ethereum/web3.py#2355 where dev depends upon previously defined directives(tester, linter, docs).

Syntax for extras_require is documented here.

It breaks if I use `open().read()` as value of a field

Example:

setup(
    ...
    long_description=open(...).read(),
    ...
)

NotImplementedError: unparsable: long_description=

It can't pars the long_description with that value. I think it should pars it to something like the following line in setup.cfg:

long_description = file: filename

"Unparsable" NotImplemementedError when getting values from variables

If setup.py gets setup() values from variables, the script blows up with a NotImplementedError:

Traceback (most recent call last):
  File "site-packages/setup_py_upgrade.py", line 125, in visit_Call
    value = ast.literal_eval(kwd.value)
  File "ast.py", line 85, in literal_eval
    return _convert(node_or_string)
  File "ast.py", line 84, in _convert
    raise ValueError('malformed node or string: ' + repr(node))
ValueError: malformed node or string: <_ast.Name object at 0x7fdf99964e10>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "setup-py-upgrade", line 8, in <module>
    sys.exit(main())
  File "site-packages/setup_py_upgrade.py", line 167, in main
    visitor.visit(tree)
  File "ast.py", line 253, in visit
    return visitor(node)
  File "ast.py", line 261, in generic_visit
    self.visit(item)
  File "ast.py", line 253, in visit
    return visitor(node)
  File "ast.py", line 263, in generic_visit
    self.visit(value)
  File "ast.py", line 253, in visit
    return visitor(node)
  File "setup_py_upgrade.py", line 127, in visit_Call
    raise NotImplementedError(f'unparsable: {kwd.arg}=')
NotImplementedError: unparsable: version=

This probably isn't fully fixable as the variable could be a constant, or it might come from a separate python file, or even from reading some data from an external file.

Support package_data with empty key and multiple values

Given setup.py:

from setuptools import setup

setup(package_data={"": ["LICENSE", "NOTICE"]})

Run:

$ setup-py-upgrade .
./setup.py and ./setup.cfg written!

Produces setup.cfg:

[options.package_data]
 =
    LICENSE
    NOTICE

Expected:

[options.package_data]
* =
    LICENSE
    NOTICE

This is similar to #4 (fixed by PR #5), and the simple case still works; this setup.py:

from setuptools import setup

setup(package_data={"": ["LICENSE"]})

Correctly produces this setup.cfg:

[options.package_data]
* =
    LICENSE

Unsupported

Instead of making user do cut and paste of ext_package, ext_modules I think the script could do , i indeed looked into the setup_py_upgrade but i was unable to understand its working due to my lack of typing syntax knowledge

Stop using deprecated 'license_file' wheel option

The keyword license_file has been deprecated by wheel in v0.32 and the wheel's behaviour for discovering license files for autoincluding them is documented in the previous link. The logic for discover license files in wheel is currently here.

I suppose that the proposed solution should only render license_files when (1) a license_file is specified in setup.py but the path is not covered by wheel globs and (2) when a license_files is explicitly specified in setup.py.

Doesn't support variables for metadata values in setup.py

This is a simplified version of the setup.py used for our Python package. We'd like to move to the declarative metadata based packaging approach. I tried this tool after watching Anthony's Youtube video on this very topic. The problem I ran into seems minor but I don't have the capacity to debug it at the moment.

#
#  Copyright (c) 2016-2021 Deephaven Data Labs and Patent Pending
#
import os
import pathlib
from setuptools.extern import packaging
from setuptools import find_packages, setup

dh_version = os.environ['DEEPHAVEN_VERSION']

setup(name='deephaven',
      version=dh_version,
      packages=find_packages(exclude=("tests",)),
      url='https://deephaven.io/',
      author='Deephaven Data Labs',
      author_email='[email protected]',
      license='Deephaven Community License',
      test_loader='unittest:TestLoader',
      classifiers=[
          'Development Status :: 2 - Pre-Alpha',
          'Intended Audience :: Developers, Data Scientists',
          'Topic :: Software Development :: Build Tools',
          'License :: Other/Proprietary License',
          'Programming Language :: Python :: 3.7',
      ],
      keywords='Deephaven Development'
      )

will except out:

/tmp setup-py-upgrade .
Traceback (most recent call last):
  File "/Users/jianfengmao/.pyenv/versions/3.7.10/lib/python3.7/site-packages/setup_py_upgrade.py", line 126, in visit_Call
    value = ast.literal_eval(kwd.value)
  File "/Users/jianfengmao/.pyenv/versions/3.7.10/lib/python3.7/ast.py", line 91, in literal_eval
    return _convert(node_or_string)
  File "/Users/jianfengmao/.pyenv/versions/3.7.10/lib/python3.7/ast.py", line 90, in _convert
    return _convert_signed_num(node)
  File "/Users/jianfengmao/.pyenv/versions/3.7.10/lib/python3.7/ast.py", line 63, in _convert_signed_num
    return _convert_num(node)
  File "/Users/jianfengmao/.pyenv/versions/3.7.10/lib/python3.7/ast.py", line 55, in _convert_num
    raise ValueError('malformed node or string: ' + repr(node))
ValueError: malformed node or string: <_ast.Name object at 0x10505c1d0>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/jianfengmao/.pyenv/versions/3.7.10/bin/setup-py-upgrade", line 8, in <module>
    sys.exit(main())
  File "/Users/jianfengmao/.pyenv/versions/3.7.10/lib/python3.7/site-packages/setup_py_upgrade.py", line 168, in main
    visitor.visit(tree)
  File "/Users/jianfengmao/.pyenv/versions/3.7.10/lib/python3.7/ast.py", line 271, in visit
    return visitor(node)
  File "/Users/jianfengmao/.pyenv/versions/3.7.10/lib/python3.7/ast.py", line 279, in generic_visit
    self.visit(item)
  File "/Users/jianfengmao/.pyenv/versions/3.7.10/lib/python3.7/ast.py", line 271, in visit
    return visitor(node)
  File "/Users/jianfengmao/.pyenv/versions/3.7.10/lib/python3.7/ast.py", line 281, in generic_visit
    self.visit(value)
  File "/Users/jianfengmao/.pyenv/versions/3.7.10/lib/python3.7/ast.py", line 271, in visit
    return visitor(node)
  File "/Users/jianfengmao/.pyenv/versions/3.7.10/lib/python3.7/site-packages/setup_py_upgrade.py", line 128, in visit_Call
    raise NotImplementedError(f'unparsable: {kwd.arg}=')
NotImplementedError: unparsable: version=

suport upgrading pbr setup.py/setup.cfg to native setuptools declarative

https://docs.openstack.org/pbr/3.0.0/#setup-cfg

support entry_points passed as a string

example setup.py:

#!/usr/bin/env python

from __future__ import annotations

import os

from setuptools import find_packages, setup

setup(
    name="distributed",
    version="0.0.0",
    description="Distributed scheduler for Dask",
    url="https://distributed.dask.org",
    project_urls={
        "Source": "https://github.com/dask/distributed",
    },
    maintainer="Matthew Rocklin",
    maintainer_email="[email protected]",
    python_requires=">=3.8",
    license="BSD",
    package_data={
        "": ["templates/index.html", "template.html"],
        "distributed": ["http/templates/*.html", "py.typed"],
    },
    include_package_data=True,
    install_requires=[],
    packages=find_packages(exclude=["*tests*"]),
    classifiers=[
        "Development Status :: 5 - Production/Stable",
        "Intended Audience :: Developers",
        "Intended Audience :: Science/Research",
        "License :: OSI Approved :: BSD License",
        "Operating System :: OS Independent",
        "Programming Language :: Python",
        "Programming Language :: Python :: 3",
        "Programming Language :: Python :: 3 :: Only",
        "Programming Language :: Python :: 3.8",
        "Programming Language :: Python :: 3.9",
        "Programming Language :: Python :: 3.10",
        "Programming Language :: Python :: 3.11",
        "Topic :: Scientific/Engineering",
        "Topic :: System :: Distributed Computing",
    ],
    entry_points="""
        [console_scripts]
        dask-ssh=distributed.cli.dask_ssh:main
        dask-scheduler=distributed.cli.dask_scheduler:main
        dask-worker=distributed.cli.dask_worker:main
        [dask_cli]
        scheduler=distributed.cli.dask_scheduler:main
        worker=distributed.cli.dask_worker:main
        ssh=distributed.cli.dask_ssh:main
        spec=distributed.cli.dask_spec:main
      """,
    # https://mypy.readthedocs.io/en/latest/installed_packages.html
    zip_safe=False,
)

results in:

Traceback (most recent call last):
  File "/home/graingert/.local/pipx/.cache/f9e756ed0e98ae6/bin/setup-py-upgrade", line 8, in <module>
    sys.exit(main())
  File "/home/graingert/.local/pipx/.cache/f9e756ed0e98ae6/lib/python3.10/site-packages/setup_py_upgrade.py", line 182, in main
    sections = {k: _reformat(v) for k, v in visitor.sections.items() if v}
  File "/home/graingert/.local/pipx/.cache/f9e756ed0e98ae6/lib/python3.10/site-packages/setup_py_upgrade.py", line 182, in <dictcomp>
    sections = {k: _reformat(v) for k, v in visitor.sections.items() if v}
  File "/home/graingert/.local/pipx/.cache/f9e756ed0e98ae6/lib/python3.10/site-packages/setup_py_upgrade.py", line 148, in _reformat
    for key, value in section.items():
AttributeError: 'str' object has no attribute 'items'

Multiple failures on running the package

When try run the setup-py-upgrade ../flake8. I am getting following errors`

→ setup-py-upgrade ../flake8
Traceback (most recent call last):
  File "/home/surya/py_venvs/flake8/lib/python3.7/site-packages/setup_py_upgrade.py", line 121, in visit_Call
    value = ast.literal_eval(kwd.value)
  File "/usr/lib/python3.7/ast.py", line 91, in literal_eval
    return _convert(node_or_string)
  File "/usr/lib/python3.7/ast.py", line 90, in _convert
    return _convert_signed_num(node)
  File "/usr/lib/python3.7/ast.py", line 63, in _convert_signed_num
    return _convert_num(node)
  File "/usr/lib/python3.7/ast.py", line 55, in _convert_num
    raise ValueError('malformed node or string: ' + repr(node))
ValueError: malformed node or string: <_ast.Attribute object at 0x7f8acddfd358>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/surya/py_venvs/flake8/bin/setup-py-upgrade", line 10, in <module>
    sys.exit(main())
  File "/home/surya/py_venvs/flake8/lib/python3.7/site-packages/setup_py_upgrade.py", line 154, in main
    visitor.visit(tree)
  File "/usr/lib/python3.7/ast.py", line 262, in visit
    return visitor(node)
  File "/usr/lib/python3.7/ast.py", line 270, in generic_visit
    self.visit(item)
  File "/usr/lib/python3.7/ast.py", line 262, in visit
    return visitor(node)
  File "/usr/lib/python3.7/ast.py", line 272, in generic_visit
    self.visit(value)
  File "/usr/lib/python3.7/ast.py", line 262, in visit
    return visitor(node)
  File "/home/surya/py_venvs/flake8/lib/python3.7/site-packages/setup_py_upgrade.py", line 123, in visit_Call
    raise NotImplementedError(f'unparsable {kwd.arg}')
NotImplementedError: unparsable version

OS: Ubuntu 19.04
Python: 3.7

Dictionaries not converted properly

Hi @asottile

Great package, using it to upgrade all my projects to declarative metadata right now.

I've had one issue with dictionaries beingn converted to their repr():

$ cat setup.py
from setuptools import setup

setup(project_urls={'homepage': 'https://adamj.eu'})
$ setup-py-upgrade .
./setup.py and ./setup.cfg written!
$ cat setup.py
from setuptools import setup
setup()
$ cat setup.cfg
[metadata]
project_urls = {'homepage': 'https://adamj.eu'}

The expected output for dictionaries is key = value, one per line, like:

[metadata]
project_urls =
    homepage = https://adamj.eu

It's not a bother for me, I can fix my projects by hand. But it does lead to a slightly confusing error which users might need a little time to debug from the message and the setup.cfg docs.

Thanks,

Adam

support for data_files kwarg

input:

setup(
      name="example",
      data_files=[('bitmaps', ['bm/b1.gif', 'bm/b2.gif']),
                  ('config', ['cfg/data.cfg'])],
     )

output:

Traceback (most recent call last):
  File "/Users/thomas/.local/pipx/.cache/f253ae1f378f350/bin/setup-py-upgrade", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Users/thomas/.local/pipx/.cache/f253ae1f378f350/lib/python3.12/site-packages/setup_py_upgrade.py", line 186, in main
    sections = {k: _reformat(v) for k, v in visitor.sections.items() if v}
                   ^^^^^^^^^^^^
  File "/Users/thomas/.local/pipx/.cache/f253ae1f378f350/lib/python3.12/site-packages/setup_py_upgrade.py", line 154, in _reformat
    new_section[key] = _list_as_str(value)
                       ^^^^^^^^^^^^^^^^^^^
  File "/Users/thomas/.local/pipx/.cache/f253ae1f378f350/lib/python3.12/site-packages/setup_py_upgrade.py", line 143, in _list_as_str
    return '\n' + '\n'.join(lst)
                  ^^^^^^^^^^^^^^
TypeError: sequence item 0: expected str instance, tuple found

data_files is deprecated though - might be nicer to get a different exception though

Support package_data with empty key

Given this minimal setup.py file (which admittedly needs more things to be filled in to be usable, but works for this example):

from setuptools import setup

setup(package_data={"": ["*.yaml"]})

setup-py-upgrade will produce the following setup.cfg:

[options.package_data]
 =
    *.yaml

However this file cannot be read by configparser or similar tools. For example:

$ pre-commit try-repo https://github.com/asottile/setup-cfg-fmt setup-cfg-fmt --files setup.cfg
===============================================================================
Using config:
===============================================================================
repos:
-   repo: https://github.com/asottile/setup-cfg-fmt
    rev: f1de3ef475160e6871983e3f0a9790b2b74b734e
    hooks:
    -   id: setup-cfg-fmt
===============================================================================
[INFO] Initializing environment for https://github.com/asottile/setup-cfg-fmt.
[INFO] Installing environment for https://github.com/asottile/setup-cfg-fmt.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
setup-cfg-fmt............................................................Failed
hookid: setup-cfg-fmt

Traceback (most recent call last):
  File "/var/folders/91/k4gwvn_n11v22vskgsvpm3fr0000gq/T/tmpdddrjpuh/repozbmcoj8d/py_env-python3/bin/setup-cfg-fmt", line 11, in <module>
    sys.exit(main())
  File "/var/folders/91/k4gwvn_n11v22vskgsvpm3fr0000gq/T/tmpdddrjpuh/repozbmcoj8d/py_env-python3/lib/python3.6/site-packages/setup_cfg_fmt.py", line 341, in main
    max_py_version=args.max_py_version,
  File "/var/folders/91/k4gwvn_n11v22vskgsvpm3fr0000gq/T/tmpdddrjpuh/repozbmcoj8d/py_env-python3/lib/python3.6/site-packages/setup_cfg_fmt.py", line 237, in format_file
    cfg.read_string(contents)
  File "/Users/mxr/.pyenv/versions/3.6.5/lib/python3.6/configparser.py", line 723, in read_string
    self.read_file(sfile, source)
  File "/Users/mxr/.pyenv/versions/3.6.5/lib/python3.6/configparser.py", line 718, in read_file
    self._read(f, source)
  File "/Users/mxr/.pyenv/versions/3.6.5/lib/python3.6/configparser.py", line 1111, in _read
    raise e
configparser.ParsingError: Source contains parsing errors: '<string>'
	[line  2]: ' =\n'
	[line  3]: '    *.yaml\n'

asottile / setup-py-upgrade Goto Github PK

setup-py-upgrade's Issues

strips shebang lines

Consider adding support for `extras_require`

It breaks if I use `open().read()` as value of a field

"Unparsable" NotImplemementedError when getting values from variables

Support package_data with empty key and multiple values

Unsupported

Stop using deprecated 'license_file' wheel option

Doesn't support variables for metadata values in setup.py

suport upgrading pbr setup.py/setup.cfg to native setuptools declarative

support entry_points passed as a string

Multiple failures on running the package

Dictionaries not converted properly

support for data_files kwarg

Support package_data with empty key

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent