pycqa / autoflake Goto Github PK

https://github.com/PyCQA/pyflakes/releases/tag/1.3.0 included support for detecting duplicate dictionary keys, strictly only reporting when the values differ, as there is no confusion in having the same key,value combination appear multiple times.

It would be nice for autoflake to implement a automatic fixer for this, removing all but the last value for a key in a dictionary.

The standard_paths function calls get_python_lib. However, it only uses the default plat_specific value (False), which means it looks in the non-platform-specific directory. This can miss many or even all python standard library packages depending on the architecture and installation details. It would be much more reliable to check both the platform-specific and platform-independent versions of the directory (so call the function with both plat_specific=False and plat_specific=True, perhaps in a loop).

If there is only one star import, from foo import * can be fixed by replacing the * with any undefined names.

This would ideally be done by using StarImportation and ImportStarUsage which were introduced PyCQA/pyflakes@0532189b3 (v1.1.0).

I am looking to find all python files in my d:\ and run autoflake on it? how would i do this?

I saw a mention of the -r parameter but it did not work for me.

A typical way to set up a project is to have the modules specify __all__ variables and then have the __init__.py from .module_name import *. This causes autoflake to remove all of the imports.

Please consider leaving the imports in __init__.py.

I have a Django app whose traditional models.py file is actually a models package. To keep it compatible with other Django functions, the package's __init__.py contains:

from .base import *

When I ran autoflake --in-place --remove-all-unused-imports src it stripped out this line, breaking my entire app, and the cause wasn't immediately obvious.

Still, autoflake correctly removed a ton of other unused imports. Is there any option to flag certain lines as exceptions, so you can run autoflake on an entire codebase but ensure that it won't incorrectly remove a line that's an exception to the rule?

Pylint supports this feature with a syntax like:

some_line # pylint: disable=rule-name

Does autoflake support anything like that?

We run autoflake with -c -r, so it checks every Python file in the repo in check mode. Our CI output looks like this:

+ autoflake -cr --remove-all-unused-imports --remove-unused-variables
--
  | No issues detected!
  | No issues detected!
  | No issues detected!
<snip hundreds of lines>
...

Our ideal behavior in check mode would be:

• print problems to stderr
• exit code set if any issues
• silence for no issues

Is there a way to accomplish this? (From looking at the code, seems that the print happens unconditionally in check mode).

If not, do you have suggestions about the user-friendliest way to add a way to accomplish it? I would be happy to make the patch.

I'm running autoflake as a hook with pre-commit:

- repo: https://github.com/myint/autoflake
  rev: v1.4
  hooks:
    - id: autoflake
      args: ['--in-place', '--remove-all-unused-imports', '--ignore-init-module-imports']

When autoflake makes some changes, all it says is that it made some changes:

autoflake................................................................Failed
- hook id: autoflake
- files were modified by this hook

but doesn't say what or where, I have to look at my unstaged files to see what happened. Since it usually doesn't make too many changes, it would be helpful if there was a flag that would print the diff and make the changes.

Can we have the ability to set up a config in a setup.cfg (which also helps it run in tandem with flake8's configuration) for which parts of the autolinter we want to run by default?

The problem occurs when I try to remove some unused imports from a multiline import like this one:

from argparse import (ArgumentParser,
                      ArgumentError,
                      ArgumentTypeError)

Even if none of the imports in the multiline import is used, autoflake doesn't remove anything.

Hi,

Would you be able to update the anaconda cloud with the latest autoflake version 1.1 so I can use conda to install autoflake instead of needing to use pip?
The page is: https://anaconda.org/auto/autoflake

Running autoflake --remove-all-unused-imports on this file

from typing import List, TYPE_CHECKING

if TYPE_CHECKING:
    from pandas import DataFrame

def foo(a: List["DataFrame"]) -> None:
    print(a[0].size)

produces

from typing import List, TYPE_CHECKING

if TYPE_CHECKING:
    pass # <----

def foo(a: List["DataFrame"]) -> None:
    print(a[0].size)

i.e. it removes the DataFrame import despite the type being used. I think this is because it is enclosed in the List because if I include "DataFrame" as a top-level type, the import is retained, as expected:

# this example stays the same
from typing import List, TYPE_CHECKING

if TYPE_CHECKING:
    from pandas import DataFrame

def foo(a: "DataFrame") -> None:
    print(a.size)

I ran into a lack of cleaning for the following which pyflakes is catching:

try:
  res = urllib2.urlopen(url, timeout=10)
except urllib2.URLError as e:
  return False

Error message looks like:

file.py:109: local variable 'e' is assigned to but never used

This is more of a suggestion than a bug. For easier review of tests, it would be a good move to have all tests be moved to input/output format.

For example, an example set would be:

# input.py
import foo
import zap
x = 1

# output.py
x = 1

This idea is taken from mustache but is widely applicable.

consider

11:03 AM noon ∈ meh.py>cat meh.py 
from math import (
        sqrt,
        log
    )

print(log(2))
11:03 AM noon ∈ meh.py>autoflake --remove-all-unused-imports meh.py 

:(

11:03 AM noon ∈ meh.py>autoflake --version
autoflake 0.6.6

I'm somewhat curious what the current limitations on star-import expansion are due to.

I haven't looked at the code in any depth, but it seems it'd be pretty straight-forward (if not simple) to fully qualify all components imported via star imports. Get the ast of the relevant code file, walk the tree, find items where the code-string doesn't reflect the actual path, and replace.

This, of course, does require building the AST of the source file, which is being processed, which seems more complex then what pyflakes does, so I'm not sure if this is out of scope for what autoflake is trying to do.

I can't describe it well in title, so I just show the example.

try:
    import playhouse.postgres_ext as pw_pext
except ImportError:
    pass

in this case, pw_pext is unused import.

After autoflake --in-place --remove-unused-variables --remove-all-unused-imports $file

try:
    pass
except ImportError:
    pass

Given the following example:

from pyspark.sql.types import (
    ArrayType,
    BooleanType,
    IntegerType,
)

a = ArrayType()

Run:

pyflakes example.py

Result:

example.py:1: 'pyspark.sql.types.BooleanType' imported but unused
example.py:1: 'pyspark.sql.types.IntegerType' imported but unused

Running

autoflake --remove-all-unused-imports example.py

Nothing happens.

autoflake 1.2
pyflakes 2.0.0

I would expect it to delete the first 'b': 1 line, but it instead deletes 'b': 6.

foo.py:

{
    'a': 0,
    'b': 1,
    'c': 2,
    'd': 3,
    'e': 4,
    'f': 5,
    'b': 6,
}

./autoflake.py --remove-duplicate-keys foo.py:

--- original/foo.py
+++ fixed/foo.py
@@ -5,5 +5,4 @@
     'd': 3,
     'e': 4,
     'f': 5,
-    'b': 6,
 }

input:

def f():
    pass


def bar():
    user = f(k='')

expected output

def f():
    pass


def bar():
    f(k='')

actual output

def f():
    pass


def bar():
    user = f(k='')

file:

# -*- coding: utf-8 -*-

import os

# Umlauts: üöäß

print 'foo'

I patched autoflake, to see the traceback. Before autoflake did silently nothing.

--- a/autoflake.py
+++ b/autoflake.py
@@ -114,7 +114,7 @@ def check(source):
         # Python 2.
         try:
             source = source.encode()
-        except UnicodeError:  # pragma: no cover
+        except IOError:#UnicodeError:  # pragma: no cover
             return []

     reporter = ListReporter()

Traceback:

modwork_eins_dmes@workdevel114:~$ autoflake -i --remove-all-unused-imports tmp/t.py 
Traceback (most recent call last):
  File "/home/modwork_eins_dmes/bin/autoflake", line 9, in <module>
    load_entry_point('autoflake==0.6.5', 'console_scripts', 'autoflake')()
  File "/home/modwork_eins_dmes/src/autoflake/autoflake.py", line 557, in main
    standard_error=sys.stderr)
  File "/home/modwork_eins_dmes/src/autoflake/autoflake.py", line 540, in _main
    fix_file(name, args=args, standard_out=standard_out)
  File "/home/modwork_eins_dmes/src/autoflake/autoflake.py", line 430, in fix_file
    remove_unused_variables=args.remove_unused_variables)
  File "/home/modwork_eins_dmes/src/autoflake/autoflake.py", line 400, in fix_code
    remove_unused_variables=remove_unused_variables))))
  File "/home/modwork_eins_dmes/src/autoflake/autoflake.py", line 232, in filter_code
    messages = check(source)
  File "/home/modwork_eins_dmes/src/autoflake/autoflake.py", line 116, in check
    source = source.encode()
UnicodeEncodeError: 'ascii' codec can't encode characters in position 37-40: ordinal not in range(128)

Now:

Running autoflake...
Unused imports/variables detected.%

How will be great:

Running autoflake...
src/foo/bar/file.py - Unused imports/variables detected.

I get UnicodeDecodeError if there is a file with unicode name.
This is not a problem for autopep8.

$ ls
é.txt  x.py
$ autopep8 . -r -i
$ autoflake . -r -i
Traceback (most recent call last):
  File "/usr/local/bin/autoflake", line 45, in <module>
    standard_error=sys.stderr))
  File "/usr/local/lib/python2.7/dist-packages/autoflake.py", line 495, in main
    if f.endswith('.py') and
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 0: ordinal not in range(128)

Currently, import from type annotations are removed.
E.g.

from typing import Any

a = None    # type: Any

becomes

a = None    # type: Any

I updated flake8 and noticed that autoflake doesn't fix F541: f-string without any placeholders. I'd be happy to write up a PR for this, but I'm not familiar with the code--anybody have pointers?

It would be nice to have it configurable what is done. Due to Issue #7 I currently don't use this program but would like to remove all unnecessary pass statements.

A integration into autopep8 might also be nice. Every case of this could get an own identifier (e.g. W1234) - so that things are unified and one doesn't need to learn a lot of different options for each tool.

First of all, kudos for the sweet tool!

To the issue at hand, it seems that import statements with comments at the end of the line are being skipped. I noticed this while trying to use autoflake to cleanup imports from OpenAPI-generated code.

Here's a sample:

foo.py (before)

import sys

# Removed (as expected)
import json
from typing import List
from typing import Dict
# Not removed!!!
import sqlite3  # foo
from datetime import date, datetime  # noqa: E501

foo = datetime()
print('hello world', foo)
sys.exit(0)

Command

autoflake --in-place --remove-all-unused-imports foo.py

This exits with a status of zero and no output.

foo.py (after)

import sys

# Removed (as expected)
# Not removed!!!
import sqlite3  # foo
from datetime import date, datetime  # noqa: E501

foo = datetime()
print('hello world', foo)
sys.exit(0)

I'm on the fence about what my expectations would be here. I think either of the following would be acceptable:

1. Do not remove the import: A warning should indicate that an unused import is still present.
2. Remove the import but leave the comment (or remaining imports) intact.

Add --exclude parameter to exclude files with a certain pattern

Introduction

Hi, I've been using autoflake for quite a while and I must to admit is a really awesome tool... so first of all, thanks for creating it :)

Here's the thing, I've always been scared of using --remove-unused-variables but today I've decided to give it a shot and test it for a bit... overall behaves fairly well but I've found some corner cases producing unoptimal code, the

Requirements

Version: 1.3.1
OS: win7
python: Python 3.8.2 (tags/v3.8.2:7b3ab59, Feb 25 2020, 22:45:29) [MSC v.1916 32 bit (Intel)] on win32

Problem

Running autoflake --remove-all-unused-imports --remove-unused-variables --ignore-init-module-imports --in-place foo.py with below code:

def foo1():
    a = 1

    return 2

def foo2():
    a = 1
    return 2

Will produce:

def foo1():
    pass

    return 2


def foo2():
    return 2

Expected output

def foo1():
    return 2


def foo2():
    return 2

Question

Why is the redundant pass hasn't been removed? Is this a bug? Is it intended behaviour? Is there any hidden directive I need to use to get rid of it? I can see there was this old issue #25 but after reading it it's still unclear to me what the final decission was...

Thanks in advance :)

Docs say:

python ¶
The hook repository must have a setup.py. It will be installed via pip install .. The installed package will provide an executable that will match the entry – usually through console_scripts or scripts in setup.py.

Support: python hooks work without any system-level dependencies. It has been tested on linux, macOS, windows, and cygwin.

But this is not entirely true. You cannot install arbitrary tools and use them with pre-commit.

The tool you install currently has to provide a .pre-commit-hooks.yaml, otherwise you get the error:

/.cache/pre-commit/repohzjhAS/.pre-commit-hooks.yaml does not exist

This seems like a design flaw.

As a user I should be able to provide the necessary configuration to use the target package as a pre-commit hook from within my own project's .pre-commit-config.yaml file, without needing any cooperation from the tool itself.

If the .pre-commit-hooks.yaml file has a role at all it can be for plugin authors to provide useful defaults. But it should not be required in order to use a particular tool.

I've tried to find a magic flag or hack that lets to run autoflake on all files in a directory. Like what flake8 and isort have. Is this something that is possible, or even desirable?

A workaround is to use:

find . -name '*.py' | xargs autoflake --in-place --remove-all-unused-imports --remove-unused-variables

But this doesn't work on windows, and requires some knowhow into how find and xargs work, something that likely many beginners lack.

Thanks for a great project!

As mentioned on #14 , pyflakes.messages.ImportStarUsage used by #18 was introduced PyCQA/pyflakes@0532189 (v1.1.0).

d756487 only increased the minimum to v1.0

cc @adhikasp

add .pre-commit-hooks.yaml to project root to support https://pre-commit.com/

-   id: autoflake
    name: autoflake
    entry: autoflake
    language: python
    'types': [python]
    require_serial: true

3d81736 is committed after latest release which is 1.3.1. Please add 1.3.2 with this feature.

The documentation is pointing to a flag --check, but trying to run autoflake --check my_file.py I receive the following error:

usage: autoflake [-h] [-i] [-r] [--exclude globs] [--imports IMPORTS]
                 [--expand-star-imports] [--remove-all-unused-imports]
                 [--remove-duplicate-keys] [--remove-unused-variables]
                 [--version]
                 files [files ...]
autoflake: error: unrecognized arguments: --check

I'm on version autoflake 1.2

Is there a known work-around for this? I need this to include it in my CI integration. I would also be fine with the regular output of autoflake my_file.py, but the error code is always 0, even if it discovers changes...

from typing import List

def hello(): # type: () -> None
    print 'hello'

class Example:
    def method(self, lst, opt=0, *args, **kwargs):
        # type: (List[str], int, *str, **bool) -> int
        print("Do stuff")

Running autoflake --remove-all-unused-imports would remove from typing import List. This should not be the case as List import is used for defining the type.

• https://www.python.org/dev/peps/pep-0484/#suggested-syntax-for-python-2-7-and-straddling-code
• mypy is a static type checker for Python and uses type annotation in comments for supporting python2.7 code. http://mypy.readthedocs.io/en/latest/python2.html
• http://blog.zulip.org/2016/10/13/static-types-in-python-oh-mypy/

I understand since this module is written in python, there must be some way to run the module programmatically (that is, not through subprocess).

I wonder if this is documented?

Also, instead of running on file, it would be better to run on string if possible :)

To ensure that you are 1:1 with pyflakes, each input file should be tested with pyflakes to assert it is failing and each output file should be tested with pyflakes to assert it is passing.

I noticed if I have code like:

from xyz import b, c, unused, a

autoflake converts it to:

from xyz import a
from xyz import b
from xyz import c

Is there anyway to keep it on one line?

from xyz import a, b, c

I am trying to package autoflake for openSUSE. However, there is no license file I can find, either in the github or tarball. In order for people to know how they can use your code it is really important to have a license file. Would it be possible to add one? Thank you.

When a module imports many names from sub-modules, and does not include any code except for those imports, the module is obviously intended to hold names.

Those names should be put into __all__.

e.g. the following causes pyflakes errors, as Foo and Bar are unused:

from a.b import Foo
from a.c import Bar

To avoid this,

from a.b import Bar
from a.c import Foo

__all__ = ('Bar', 'Foo')

not all pass statements occurring in places where they are not technically required are useless.

consider for instance a command line app written using the popular click library, featuring some subcommands (see docs). for the sake of argument, consider a hypothetical git-like application supporting a command like git remote add, which would look like this:

@click.group()
def remote_group(...):
    pass

@remote_group.command():
def add(...):
    ...  # implementation goes here

now, let's add docstrings, which in the case of click, also appear in --help output:

@click.group()
def remote_group(...):
    """
    Manage set of tracked repositories.
    """
    pass

@remote_group.command()
def add(...):
    """
    Add a new remote.
    """
    ...  # implementation goes here

technically, the presence of a docstring means the pass is not needed to make it valid python. but the pass shows the intent to the reader. without it i would immediately wonder if someone forget to write some code, or if some git merge went wrong. in this specific example, click actually supports shared code in functions decorated with click.group(). the presence of pass signals that this function does not need that, and is intended as an empty body.

so, in short: not all pass statements that are strictly technically unnecessary are to be considered ‘useless’.

my suggestion would be to treat the ‘function body, with only a docstring and a pass statement’ as an extra special case, and always leave the pass statements in place.

flake8-comprehensions is a lovely plugin to warn about inefficient creation of and operations on collections - and each of the warnings suggests a simple rewrite rule to resolve them. I'm not sure if this is really in scope for autoflake, but thought it worth suggesting an optional flag to do so 😄

Thanks for this fantastic tool - I've noticed there's a .pre-commit-hooks.yaml file on master, but not in the latest release.

Sorry if this comes across as rude or as a self-entitled request, I just think it would be fantastic if you could make a new release so we can use this as a pre-commit hook.

Currently autoflake reads from a file by taking a filename, and outputs a diff with a patch to make. This works well for some command line usage, but doesn't integrate so well with replacing a buffer in Vim. This could be improved by offering an option to read the file from stdin, and an option for outputting the entire file contents, with the patch applied, to stdout. stdin support could either by via flag, or by reading a special filename -. Supposing there was a flag for stdout support, then autoflake could automatically remove unused imports in Vim with both flags like so.

silent 0,$!autoflake --stdin --output-file

That command would take the contents of the current buffer, pass it to autoflake via stdin, get the new file after applying the patch back out again, and replace the current buffer with that text.

At the moment, I shall have to save my current buffer to a file, run autoflake on that temporary file and save the diff to another temporary file, and then use the :diffpatch command to apply the patch to the current buffer, in order to integrate autoflake with Vim.

Requirements

Version: 1.3.1
OS: win7
python: Python 3.8.2 (tags/v3.8.2:7b3ab59, Feb 25 2020, 22:45:29) [MSC v.1916 32 bit (Intel)] on win32

Problem

Running autoflake --remove-all-unused-imports --remove-unused-variables --ignore-init-module-imports --in-place foo.py with below code:

def foo():
    try:
        0 / 0
    except Exception as e:
        pass

Will produce:

def foo():
    try:
        0 / 0
    except Exception as e:
        pass

Expected output

def foo():
    try:
        0 / 0
    except Exception:
        pass

The use of parenthesis causes the --remove-all-unused-imports to miss unused imports.

e.g.

$ echo -e "from os import path" > example.py
$ cat example.py
from os import path
$ autoflake --in-place --remove-all-unused-imports example.py
$ cat example.py
$ echo -e "from os import (\npath\n)" > example.py
$ cat example.py
from os import (
path
)
$ python example.py
$ autoflake --in-place --remove-all-unused-imports example.py
$ cat example.py
from os import (
path
)

Since parens are legal import syntax, I would expect their presence of absence to have no effect on unused import checking.

In fix_code, filter_useless_pass is always called.

But, it would be nice to be able to remove unused import statements without modifying pass statements, and to control removal of pass statements separately.

I guess the following would fall under the same category, I was wondering is these are automatable or planned to be done in future releases.

• Multiple spaces before keyword
• Missing whitespace after keyword
• no space space around keyword
• unexpected spaces around keyword
• trailing whitespaces
• too many blank lines
• expected 2 blank lines, found 1