sqlalchemyorg / zimports Goto Github PK

View Code? Open in Web Editor NEW

73.0 73.0 5.0 136 KB

Python import rewriter

License: MIT License

Python 100.00%

zimports's Introduction

SQLAlchemy Website

This site uses zeekofile to build a static site, which can also be served locally using zeekofile.

Building and Testing:

#!/bin/bash

virtualenv .venv
source .venv/bin/activate
pip install git+https://github.com/sqlalchemyorg/zeekofile
git clone https://github.com/sqlalchemyorg/sqlalchemyorg
cd sqlalchemyorg/
pip install -r requirements.txt
zeekofile --serve

Then view:

http://127.0.0.1:8080

zimports's People

Contributors

Stargazers

Watchers

Forkers

haomingw lvt-mm schmir rafayghafoor sitedata

zimports's Issues

Allow splitting a single import over multiple lines

If an import exceeds the character limit, black will split it to multiple lines:

from .tag_genome_object_ranking_dataset_reader import TagGenomeObjectRankingDatasetReader

becomes

from .tag_genome_object_ranking_dataset_reader import (
    TagGenomeObjectRankingDatasetReader,
)

but then zimports will change it back to the one-line format. This makes black and zimports incompatible. It would be nice if zimports could allow that kind of formatting.

Release 0.6.0 references version 0.5.1

The latest release doesn't match the version listed in the setup.cfg.

zimports/setup.cfg

Line 3 in 7e97d02

version = 0.5.1

Feature Request: Ignore entire file for sorting

It would be nice to be able to mark an entire file for "no sorting" by zimports. This could either be as a directive in a setup.cfg, or as a comment in the file itself.

Do not work with pre-commit

I did not find the way to run with pre-commit hooks.

[INFO] Initializing environment for https://github.com/sqlalchemyorg/zimports.
An unexpected error has occurred: CalledProcessError: Command: ('/usr/libexec/git-core/git', 'checkout', '')
Return code: 128
Expected return code: 0
Output: (none)
Errors:
fatal: empty string is not a valid pathspec. please use . instead if you meant to match all paths

crashes on TYPE_CHECKING block that has no imports

import typing


if typing.TYPE_CHECKING:
    print("hi")

Traceback (most recent call last):
  File "/home/classic/.venv3/bin/zimports", line 33, in <module>
    sys.exit(load_entry_point('zimports', 'console_scripts', 'zimports')())
  File "/home/classic/dev/zimports/zimports/cli.py", line 110, in main
    run_with_options(options)
  File "/home/classic/dev/zimports/zimports/zimports.py", line 854, in run_with_options
    _run_file(options, filename)
  File "/home/classic/dev/zimports/zimports/zimports.py", line 798, in _run_file
    result, stats = Rewriter(options, filename, source_lines).rewrite()
  File "/home/classic/dev/zimports/zimports/zimports.py", line 160, in rewrite
    rewritten = self._do_rewrite(self.source_lines, type_check_pass=True)
  File "/home/classic/dev/zimports/zimports/zimports.py", line 103, in _do_rewrite
    imports, warnings, lines_with_code = _parse_toplevel_imports(
  File "/home/classic/dev/zimports/zimports/zimports.py", line 459, in _parse_toplevel_imports
    tree = ast.parse(source, filename)
  File "/opt/python-3.10.0/lib/python3.10/ast.py", line 50, in parse
    return compile(source, filename, mode, flags,
  File "test3.py", line 5
    print("hi")
IndentationError: unexpected indent

Fix typing issues

Some issues when using types in code have come up:

Possibility to retain `from module import x, y` permitted by PEP8

Would it be possible to retain the example in PEP8 as it is, perhaps optionally:

from subprocess import Popen, PIPE

and not breaking them up as:

from subprocess import Popen
from subprocess import PIPE

Incorrect order with pycharm style

I am running zimports --multi-imports --style pycharm --black-line-length 88 (version 0.6.0)

getting:

from sqlalchemy import (
    Column,
    DateTime,
    func,
    insert,
    Integer,
    MetaData,
    select,
    String,
)

expected:

from sqlalchemy import (
    Column,
    DateTime,
    Integer,
    MetaData,
    String,
    func,
    insert,
    select,
)

I tried to debug a little bit and I think it goes wrong when the file gets parsed from ast. Seems that it returns the nodes in the tree with its own order and then no sort is happening from zimports, but im not sure.

Add support for additional configuration files

This is a TODO in the README, but I thought an explicit issue might be helpful. From the Flake8 docs, this could be "in your project in one of setup.cfg, tox.ini, or .flake8" (there's an open issue for pyproject.toml).

Maybe the flake8.options.config.ConfigFileFinder would be useful?

Preserving comments on import lines? (e.g., `# type: ignore`)

Currently, I have a few import lines that are annotated with # type: ignore for mypy, e.g.,:

from boto3.s3.transfer import TransferConfig  # type: ignore
from boto3.session import Session  # type: ignore
import botocore  # type: ignore
import click

Unfortunately, zimports removes those comments and then mypy complains (obviously it'd be better to stub these libraries, but that's somewhat separate). Is there an option to disable this behavior?

I tried doing # noqa # type: ignore but the latter part still gets removed. I'm currently setting them in an if True block, but that feels awfully gross. :/ Appreciate any guidance (and thanks)!

IndexError when running zimports on an empty file

zimports 0.2.1 crashes with an IndexError when trying to read an empty file. This is a regression from 0.2.0:

ralf@triton ~/t % touch __init__.py
ralf@triton ~/t % zimports __init__.py 
Traceback (most recent call last):
  File "/home/ralf/.local/bin/zimports", line 8, in <module>
    sys.exit(main())
  File "/home/ralf/.local/pipx/venvs/zimports/lib64/python3.8/site-packages/zimports.py", line 766, in main
    _run_file(options, filename)
  File "/home/ralf/.local/pipx/venvs/zimports/lib64/python3.8/site-packages/zimports.py", line 622, in _run_file
    lines, encoding_comment = _read_python_source(filename)
  File "/home/ralf/.local/pipx/venvs/zimports/lib64/python3.8/site-packages/zimports.py", line 560, in _read_python_source
    if text[-1] == "\n":
IndexError: string index out of range

failures in python 3.10

Traceback (most recent call last):
  File "/home/classic/dev/zimports/tests.py", line 118, in test_whitespace3
    self._assert_file("whitespace3.py")
  File "/home/classic/dev/zimports/tests.py", line 56, in _assert_file
    zimports.main(
  File "/home/classic/dev/zimports/zimports/cli.py", line 110, in main
    run_with_options(options)
  File "/home/classic/dev/zimports/zimports/zimports.py", line 842, in run_with_options
    _run_file(options, filename)
  File "/home/classic/dev/zimports/zimports/zimports.py", line 786, in _run_file
    result, stats = Rewriter(options, filename, source_lines).rewrite()
  File "/home/classic/dev/zimports/zimports/zimports.py", line 161, in rewrite
    rewritten = self._do_rewrite(rewritten, type_check_pass=False)
  File "/home/classic/dev/zimports/zimports/zimports.py", line 103, in _do_rewrite
    imports, warnings, lines_with_code = _parse_toplevel_imports(
  File "/home/classic/dev/zimports/zimports/zimports.py", line 452, in _parse_toplevel_imports
    tree = ast.parse(source, filename)
  File "/opt/python-3.10.0/lib/python3.10/ast.py", line 50, in parse
    return compile(source, filename, mode, flags,
  File "test_files/whitespace3.py", line 9
    
    ^
IndentationError: unexpected indent
=================================================================== short test summary info ====================================================================
FAILED tests.py::ImportsTest::test_per_file_ignore -   File "test_files/tricky_parens.py", line 59
FAILED tests.py::ImportsTest::test_per_file_ignore_file_not_selected -   File "test_files/tricky_parens.py", line 59
FAILED tests.py::ImportsTest::test_per_file_ignore_other_codes -   File "test_files/tricky_parens.py", line 59
FAILED tests.py::ImportsTest::test_sqla_test_file -   File "test_files/sqla_test_file.py", line 82
FAILED tests.py::ImportsTest::test_tricky_parens -   File "test_files/tricky_parens.py", line 59
FAILED tests.py::ImportsTest::test_type_checking2 -   File "test_files/type_checking2.py", line 41
FAILED tests.py::ImportsTest::test_whitespace2 -   File "test_files/whitespace2.py", line 6
FAILED tests.py::ImportsTest::test_whitespace3 -   File "test_files/whitespace3.py", line 9

another weird conditional issue

not sure what is special here

import typing
from typing import TypeVar
from typing import Any

if typing.TYPE_CHECKING:
    from .state import InstanceState  # noqa

_T = TypeVar("_T", bound=Any)

syntax error

Smarter discovery of the configuration file

At the moment zimports looks for the configuration file in the current directory. That results in zimports giving different results based on the fact if it's able to load the settings from the current directory or not (e.g. for a src based layout zimports src may give different results from cd src; zimports .)

I think what it should do, is try to find setup.cfg for each argument passed in parent
directories of that argument (probably stopping when it finds the root of a git/mercurial/.. repo) or a setup.cfg file. Read that file and merge the options with those given on the command line and use that to sort imports.

If you think this way to discover the setup.cfg file is fine, I can contribute a PR.

Allow multi-imports for typing

Per the Google Style Guide, an exception to the "one import per line" rule is allowed for the typing module. Currently, it looks like the only way to accomplish this is with --multi-imports, which affects all imports.

IE, instead of:

from typing import Dict
from typing import IO
from typing import Optional
from typing import Sequence
from typing import Tuple
from typing import Union

Format as:

from typing import Dict, IO, Optional, Union, Sequence, Tuple

Allows sorting `if TYPE_CHECKING:` blocks

Currently zimports sorts only module level imports without indentation. Since when using type annotations importing some symbols in a guarded block is common, it would be nice if these imports could be sorted

Add option to parallelize files

Since to support typing zimports has to do 3 passes for each file, it's not the fastest, and sqlalchemy has a lot of files.

Probably on by default with option to disable.

File encoding issue

Operating System: Window 10
pre-commit version: 2.5.1

Earlier today i made a Pull Request to fix a few typos i found inside 5 SQLAlchemy files. I have pre-commit installed so that black formatting, flake8 and zimports checks are made before my commit.

However, the zimports check raised the below error when i tried to make a commit with the file SQLAlchemy file test_quote.py:

zimports.................................................................Failed
- hook id: zimports
- exit code: 1
Traceback (most recent call last):
  File "c:\users\owner\appdata\local\programs\python\python37\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\owner\appdata\local\programs\python\python37\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\Owner\.cache\pre-commit\repov34h3x3z\py_env-python3.7\Scripts\zimports.EXE\__main__.py", line 7, in <module>
  File "c:\users\owner\.cache\pre-commit\repov34h3x3z\py_env-python3.7\lib\site-packages\zimports.py", line 698, in main
    _run_file(options, filename)
  File "c:\users\owner\.cache\pre-commit\repov34h3x3z\py_env-python3.7\lib\site-packages\zimports.py", line 561, in _run_file
    source_lines = [line.rstrip() for line in file_]
  File "c:\users\owner\.cache\pre-commit\repov34h3x3z\py_env-python3.7\lib\site-packages\zimports.py", line 561, in <listcomp>
    source_lines = [line.rstrip() for line in file_]
  File "c:\users\owner\appdata\local\programs\python\python37\lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 1024: character maps to <undefined>

After some investigating i found out that because i am on a windows PC and no encoding is specified in zimports.py file the encoding defaults to the systems locale, CP-1252. CP-1252 doesn't contain byte 0x90 thus raising the error.

The workaround i used to resolve this was to make the following edit to line 560 in the zimports.py file (inside the pre-commit cache folder that gets generated):

##\.cache\pre-commit\repov34h3x3z\py_env-python3.7\lib\site-packages\zimports.py"

def _lines_with_newlines(lines):
    for line in lines:
        yield line + "\n"


def _run_file(options, filename):
-    with open(filename) as file_:
+    with open(filename, encoding="utf8") as file_:

So my question is should any changes be made to specify the file encoding explicitly rather than using the default as it currently is? If yes can i make the PR?

I can see that the python community might be moving towards using utf8 as the default encoding in the future

Thanks,

Ramon

detect Python source file encoding including magic comment properly, use it explicitly when file is read and written to.

Hi,

I am running this with pre-commit (in the process of setting it up) and encountering issues where the file i'm running it on has a "# -- coding: utf-8 --" up the top, but in the zimports script it uses with open(filename) as file_ in _run_file which results in the platform default and a UnicodeDecodeError.

I take it this isn't completely unicode safe yet?

Is there some way to not change the order of the import and the code before that import?

Sorry for my English. The code is more obvious.

In Django I need a line django.setup() before importing django model for some scripts beyond the django project:

import django
django.setup()

from apps.model import Blah

zimports will change it to

import django
from apps.model import Blah

django.setup()

What is the right way in this situation?

crashes on TYPE CHECKING with ELSE

e.g.

if TYPE_CHECKING:
   # etc.
else:
   # etc.

rewrites "import x # type: ignore # noqa as "# noqa"

e.g.

import greenlet # type: ignore #noqa

blows away the "type ignore" part

zimports cannot read from /dev/stdin anymore

When calling zimports with /dev/stdin as input file, zimports 0.2.1 crashes with OSError: [Errno 29] Illegal seek
This is a regression from 0.2.0.

ralf@triton ~/t % cat zimports.py|zimports --stdout /dev/stdin
Traceback (most recent call last):
  File "/home/ralf/.local/bin/zimports", line 8, in <module>
    sys.exit(main())
  File "/home/ralf/.local/pipx/venvs/zimports/lib64/python3.8/site-packages/zimports.py", line 766, in main
    _run_file(options, filename)
  File "/home/ralf/.local/pipx/venvs/zimports/lib64/python3.8/site-packages/zimports.py", line 622, in _run_file
    lines, encoding_comment = _read_python_source(filename)
  File "/home/ralf/.local/pipx/venvs/zimports/lib64/python3.8/site-packages/zimports.py", line 558, in _read_python_source
    encoding_comment = _parse_magic_encoding_comment(file_)
  File "/home/ralf/.local/pipx/venvs/zimports/lib64/python3.8/site-packages/zimports.py", line 581, in _parse_magic_encoding_comment
    pos = fp.tell()
OSError: [Errno 29] Illegal seek

I'm using this in my zimports.el emacs package