aroberge / friendly Goto Github PK

As mentioned in issue #8, Thonny could possibly make use of the information provided by friendly-traceback if it were available in structured form. Currently Thonny's GUI part gets something like this as input:

{
    'intro' : 'General explanation of the error message itself',
    'possible_causes' : [
        {
            'title' : 'eg. Did you misspell the variable?', 
            'text' : 'longer text, may contain several paragraphs',
            'weight' : 3 // how likely this cause is
        }, 
        { 
            ...
        }
    ]
}

While Friendly-traceback aims to normally present the information a simple text with no fancy formatting, except for respecting included line breaks and indentation, it could be useful to have an intermediate step where the information is recorded in a structured way inspired by the above. In addition to the current formatting done by Thonny, this might make it easier for other environment to customize the look of the output - for example, by adding colours.

The revised check_syntax included in version 0.0.27 ignores the lang parameter.

check_syntax allows to see if syntax errors exist in a file or a source (string), providing a friendly-traceback explanation of the first error found. This is described in issue #26 . It might be useful to have a similar function that would run such code, to see if exceptions are raised when a given script is executed.

This is different than executing the usual way, from the command line.

Like the title said. This might make it easier to use as a plugin in various editors, when one needs more explanation about a syntax error possibly flagged by the editor.

See if all the examples given in https://realpython.com/invalid-syntax-python/ can be properly covered.

• Missing comma in dict

  • See if the same can be done for tuples, sets and list

• Can't assign to literal: already implemented

  • However, fix something like 1=2.

• Can't assign to function call

• Misspelling a keyword

• missing keyword

• 'break' outside loop

• 'continue' not properly in loop

• assign to keyword: already implemented

• attempt to use function name as keyword

• mismatched quote: message = 'don't

• EOL while scanning string literal: already implemented

• f-string: unterminated string

• Check cases like

def foo():
    return [1, 2, 3

print(foo())

or

def foo():
    return [1, 2, 3,

print(foo())

• ages = {'pam'=24}
• IndentationError: already implemented
• TabError: already implemented
• def fun();: already implemented
• fun(a=1, 2) positional argument follows keyword argument
• Missing parentheses in call to 'print'. Did you mean print('hello')?
• TypeError: 'tuple' object is not callable; note that SyntaxWarning issue in python 3.8. See if we can use this.

A common error for beginners seems to be trying to use pip inside a Python repl. Chances are that this will never occur in an environment where friendly-traceback is used, but it might still be worth adding this case as it would be easy to identify.

When adding or modifying test cases or translations, I almost always run some scripts to update the documentation and visually inspect the changes. This works well in terms of quality assurance but tends to clutter commits and make inspection of changes using tools such as gitk or Sublime Merge more tedious than it should be.

I have thus decided to more the documentation to a separate repository (visible at https://aroberge.github.io/friendly-traceback-docs/docs/html/).

However, updating the documentation using the scripts included in this repository assumes that both this repository and the documentation directory are found at the same level on a local computer, with hard-coded names:

    base_path/
            +- friendly-traceback/
            +- friendly-traceback-docs/
```

Currently, the default is to import a script instead of running it as main. This should be changed so that the default is to run a script as main, and have the option to only import it instead.

Many of the line analyzers have been created with the assumption that users would write clauses that begin a block all on the same line which should thus end with a colon. Therefore, the absence of a colon at the end would signal a mistake. They need to be rewritten, to deal properly with situtations where statements span multiple lines.

Reproduer:

print len("Pouette")

Expected: The thing about print now being a function.

Got:

$ python -m friendly_traceback repro.py

    Python exception:
        SyntaxError: invalid syntax

    A SyntaxError occurs when Python cannot understand your code.

    Python could not understand the code in the file
    'repro.py'
    beyond the location indicated below by --> and ^.

    -->1: print len("Pouette")
                ^

    I don't have enough information from Python:
        Currently, I cannot guess the likely cause of this error.
        Try to examine closely the line indicated as well as the line
        immediately above to see if you can identify some misspelled
        word, or missing symbols, like (, ), [, ], :, etc.

        You might want to report this case to
        https://github.com/aroberge/friendly-traceback/issues

I'm using Python 3.8.3.

Was going to try to PR it myself but did #48 instead, and now I have to sleep :(

File "./bbq.py", line 160
    for config_file in config_files
                                  ^
 SyntaxError: invalid syntax

from: https://mail.python.org/archives/list/[email protected]/thread/RMAZTJ3RO5L5RNHGTAPVE4UL2GXIWTSO/#ILJNAN4E5VROSODWO2UWJDHP5DCVM56G

Currently, custom formatters are allowed, but they must output a string. It should be possible to bypass this, and have a dict being returned.

A fresh install the package (07c31f7948fda4cb748d8161f329f25277f2e661) reports a problem:

Problem: source of <stdin> is not available

Here's the full stack:

misc-projects for romunov at mucek in ~/biotools  
$ git clone [email protected]:aroberge/friendly-traceback.git
Cloning into 'friendly-traceback'...
remote: Enumerating objects: 2108, done.
remote: Total 2108 (delta 0), reused 0 (delta 0), pack-reused 2108
Receiving objects: 100% (2108/2108), 1.69 MiB | 2.77 MiB/s, done.
Resolving deltas: 100% (1512/1512), done.
misc-projects for romunov at mucek in ~/biotools  
$ cd friendly-traceback/
misc-projects for romunov at mucek in ~/biotools/friendly-traceback on master 
$ pip install .[package,test,help]
Processing /home/romunov/biotools/friendly-traceback
  WARNING: friendly-traceback 0.0.10a0 does not provide the extra 'help'
  WARNING: friendly-traceback 0.0.10a0 does not provide the extra 'package'
  WARNING: friendly-traceback 0.0.10a0 does not provide the extra 'test'
Building wheels for collected packages: friendly-traceback
  Building wheel for friendly-traceback (setup.py) ... done
  Created wheel for friendly-traceback: filename=friendly_traceback-0.0.10a0-cp37-none-any.whl size=42353 sha256=146baf07efd3299cdd7ca15d3979ad6208460ae36f91695fda4c17014784069d
  Stored in directory: /home/romunov/.cache/pip/wheels/5f/97/68/27496f9bdc55bd50ee5398dca45206676aef45bb42e9971b15
Successfully built friendly-traceback
Installing collected packages: friendly-traceback
Successfully installed friendly-traceback-0.0.10a0
misc-projects for romunov at mucek in ~/biotools/friendly-traceback on master 
$ python
Python 3.7.4 (default, Jul  9 2019, 16:32:37) 
[GCC 9.1.1 20190503 (Red Hat 9.1.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import friendly_traceback
>>> a = [1,2,3]
>>> a[3]
Problem: source of <stdin> is not available
Problem: source of <stdin> is not available

    Python exception:
        IndexError: list index out of range
        
    An IndexError occurs when you are try to get an item from a list,
    a tuple, or a similar object (sequence), by using an index which
    does not exists; typically, this is because the index you give
    is greater than the length of the sequence.
    Reminder: the first item of a sequence is at index 0.
    
    Likely cause:
        In this case, the sequence is a list.
        
    Execution stopped on line 1 of file '<stdin>'.

Currently, we typically have 4 lines of code displayed, showing the context in which a given exception is raised. Here's a simple copy-pasted example:

   4: def test_import_error():
   5:     try:
-->6:         from math import Pi
   7:     except Exception:

The last line adds no useful information: it should simply be dropped.

Many error messages have changed in Python 3.8. Some are simply typos, whereas others are real improvements, like changing SyntaxError: invalid syntax to something much more precise and useful. Right now, it is a tedious job to check if particular messages have changed, and is something that can be easily overlooked. This process should be automated.

Current plan:

• Write a program goes through each unit test file, record the name of the file (or, if there are multiple test functions, the name of the function as well) together with the error message generated, and possibly the offending line that was identified, saving the result in a different file for each Python version. It might make sense to save the result as a Python dict in each file.
• Write a second program that goes through these files, and extract what changed between Python version. This final result is what should go in the documentation.

Python 3.8 is sometimes printing some Syntax Warnings. These can contain some useful information (which might be helpful) but they do add unfriendly noise. I should look into ways to capture the information and use it, without distracting the end user.

With the experimental version of Avant-Idle, it is possible to show the same traceback multiple times, by changing the verbosity level. However, the traceback thus shown has the language "hard-coded".

By adding an extra step between the extraction of the information and the formatting, it might be possible to show a given traceback in different languages.

OverflowError exception friedly-traceback

In the documentation, instead of mentioning to run

pytest

suggest instead

python -m pytest

to ensure that the correct version is run.

Adding a new exception - recording it to avoid anyone else wasting their time.

In hackinscience.org when I check the student code I run:

proc = subprocess.run(
    [
        "python3",
        "-m",
        "friendly_traceback",
        "--formatter",
        "correction_helper.friendly_traceback_markdown",
        "solution.py",
    ],
    stdout=subprocess.PIPE,
    stderr=subprocess.PIPE,
    universal_newlines=True,
)

Which give them, on error, a nice Markdown output that get properly rendered and colored in the front-end, lovely.

But for some exercises I would like to pass them arguments, but it does not look supported yet. I may have time to take a look at it by myself, but probably not today, so as a first step I'm writing this issue :)

With

$ cat reproducer.py 
"a"()

I'm getting:

$ python -m friendly_traceback fail.py 
fail.py:1: SyntaxWarning: 'str' object is not callable; perhaps you missed a comma?
  "a"()

    Python exception:
        TypeError: 'str' object is not callable
        
    A TypeError is usually caused by trying
    to combine two incompatible types of objects,
    by calling a function with the wrong type of object,
    or by tring to do an operation not allowed on a given type of object.
    
    Likely cause based on the information given by Python:
        I suspect that you had an object of this type, <a string ('str')>,
        followed by what looked like a tuple, '(...)',
        which Python took as an indication of a function call.
        Perhaps you had a missing comma before the tuple.
        
    Execution stopped on line 1 of file 'fail.py'.
    
    -->1: "a"()

Notice the <a string ('str')> which looks like an HTML anchor.

It still not searched where it comes from, but I notice it may be on purpose as a grep give me:

tests/except/test_type_error.py:        assert "I suspect that you had an object of this type, <a tuple>," in result
tests/except/test_type_error.py:        assert "I suspect that you had an object of this type, <a list>," in result

I may be biased towards Markdown as I use Markdown to render it (that's where it bite me: markdown allows HTML, so this gets parsed as an anchor without hyper reference), but I think it would be nicer as:

I suspect that you had an object of this type, a `str`,

Lang option work only with develop mode, I had this into setup.py to package data:

package_data={"gettext": ["friendly_traceback/locales/*"]},
include_package_data=True,

@ntoll, @tmontes, @aivarannamaa (and anyone else interested):

I have added some information about Mu and Thonny in the documentation for Friendly-traceback.
(See https://aroberge.github.io/friendly-traceback-docs/docs/html/mu_thonny.html as the starting point; there are three pages in total.)

Could you have a look to make sure that I do not misrepresent either Mu or Thonny or both?

And, while I have your attention ;-), feel free to read and comment some thoughts I have about the next steps for Friendly-traceback, as I briefly describe them below.

I believe it is Aivar that mentioned to me first that it would be useful to avoid duplication in projects dealing with helping users to understand tracebacks. Right now, I see Friendly-traceback as the natural project to do so. However, I want to make sure that it can be "neutral" and easily adaptable by tools like Mu and Thonny.

Seeing Friendly-traceback in action within Thonny made me appreciate even more Thonny's Assistant. I think that it would make sense to add similar analysis to Friendly-traceback. However, in many instances, there would be too much information to show in the traceback: Thonny's approach to have an additional sidebar is, in my opinion, the right way to proceed.

As a proof of concept, I am thinking of adding more "levels" to Friendly traceback, with some dedicated to providing the kind of information found currently in Thonny's Assistant. The way I would do this (as a proof of concept) would be to have a pop-up window with the information shown like it is done in Thonny's Assistant. The trick would be to design the API such that either Thonny or Mu could package the information as they see fit.
For example, Thonny could use the same sidebar it currently uses for its Assistant while, perhaps Mu could embed the information like it does
when one uses its code checker:

Of course, all the information available would support translation into other languages. Translating all the content from Friendly-traceback (with much more content yet to come) will require a huge amount of work. Furthermore, Friendly-traceback has not been "road-tested" with actual beginners. I suspect that some, possibly many of the existing explanations will have to be rewritten after I get feedback from actual users. However, this is something I really look forward to.

I made many changes to the code without making sure along the way that the demos were still working ... and they are not anymore.

Running a program and dropping into interactive mode does not work anymore:

 python -im friendly_traceback test.py

    Python exception:
        ImportError: Error while finding module specification for 'test.py' (ModuleNotFoundError: __path__ attribute not found on 'test' while trying to find 'test.py')

Some modules in the Python standard library have their own exceptions, such as decimal or turtle. After the basic exceptions are included, it might make sense to include such exceptions as these modules might be used by beginners. In particular, the turtle module would like benefit from having translations.

Please add your own examples from the standard library as a comment to this issue.

The line analyzer (line_analyzer.py) attempts to find the cause of a SyntaxError by breaking a line into tokens and going through a series of cases based on token patterns. In doing so, it completely ignores the information given by Python about the exact location where the error has been found to occur.

In some cases, it might be much more efficient to make use of the information given by Python. Furthermore, it might help to prevent misidentifying causes of errors.

Currently, information about language and verbosity level is read from either a configuration file or an environment variable set up. This means that whenever friendly-traceback is imported, some defaults are set. This can be undesirable if one simply wants to use a single function, like check_syntax, without replacing the exception hook globally.

Configuration parameters would be better left to any other programs (especially GUI) that make use of friendly-traceback and have their own configuration file.

• Remove the code
• Remove the documentation about configuration variables.

It should be possible to view/print again a friendly traceback, but with a different level.

See discussion on https://mail.python.org/archives/list/[email protected]/message/WWPXZFCCFH7UF3Q52EONRASPQOQQD2OH/

See https://unicode-table.com/en/sets/quotation-marks/ for a list.

= = =
Case reopened following this comment: https://mail.python.org/archives/list/[email protected]/message/J3ZYNUDTYEJJ6QSBVAAQKS5IISHR6XQY/

While the author of that comment is not familiar with the main target audience of friendly-traceback, he makes some good points.

For simplicity, I reproduce here the essential part of that comment:

Perhaps something like:

"""
SyntaxError: invalid character in identifier
-->2: print(“Squares:”)
^
An identifier contains one or more unallowed Unicode characters.
It is likely the "curly quote" unicode quotation mark, rather than
the required ASCII single or double quote was used for a string.
"""

See https://github.com/aroberge/friendly-traceback/projects/1 for details.

Recording this issue in case anyone else was thinking of working on it.

While figuring out what was the cause of SyntaxError: invalid syntax can be nearly hopeless, many causes of SyntaxError actually have a more useful message. The Python repository contains a useful test file (https://github.com/python/cpython/blob/master/Lib/test/test_syntax.py) that can be used as a source of cases to consider.

Below are examples extracted from that file.
= = =

• parameter and global

>>> def f(x):
...     global x
Traceback (most recent call last):
SyntaxError: name 'x' is parameter and global

• obj.None

>>> obj.None = 1
Traceback (most recent call last):
SyntaxError: invalid syntax

• assign to None

>>> None = 1
Traceback (most recent call last):
SyntaxError: cannot assign to None

• obj.True

>>> obj.True = 1
Traceback (most recent call last):
SyntaxError: invalid syntax

• assign to True

>>> True = 1
Traceback (most recent call last):
SyntaxError: cannot assign to True

• augmented assignement to True

>>> (True := 1)
Traceback (most recent call last):
SyntaxError: cannot use named assignment with True

• obj.__debug__ = 1

>>> `obj.__debug__` = 1
Traceback (most recent call last):
SyntaxError: cannot assign to __debug__

• __debug__ = 1

>>> __debug__ = 1
Traceback (most recent call last):
SyntaxError: cannot assign to __debug__

• __debug__ = 1

>>> (__debug__ := 1)
Traceback (most recent call last):
SyntaxError: cannot assign to __debug__

• assign to function call

>>> f() = 1
Traceback (most recent call last):
SyntaxError: cannot assign to function call

• delete function call

>>> del f()
Traceback (most recent call last):
SyntaxError: cannot delete function call

• assign to operator

>>> a + 1 = 2
Traceback (most recent call last):
SyntaxError: cannot assign to operator

• assign to generator expression

>>> (x for x in x) = 1
Traceback (most recent call last):
SyntaxError: cannot assign to generator expression

• assign to literal

>>> 1 = 1
Traceback (most recent call last):
SyntaxError: cannot assign to literal

• assign to literal

>>> "abc" = 1
Traceback (most recent call last):
SyntaxError: cannot assign to literal

• assign to literal

>>> b"" = 1
Traceback (most recent call last):
SyntaxError: cannot assign to literal

• assign to ellipsis

>>> ... = 1
Traceback (most recent call last):
SyntaxError: cannot assign to Ellipsis

• deprecated backtick

>>> `1` = 1
Traceback (most recent call last):
SyntaxError: invalid syntax

• assign to literal (inside tuple)

>>> (a, "b", c) = (1, 2, 3)
Traceback (most recent call last):
SyntaxError: cannot assign to literal

• assign to True (inside Tuple)

>>> (a, True, c) = (1, 2, 3)
Traceback (most recent call last):
SyntaxError: cannot assign to True

• assign to __debug__ inside tuple

>>> (a, __debug__, c) = (1, 2, 3)
Traceback (most recent call last):
SyntaxError: cannot assign to __debug__

• assign to True (inside Tuple)

>>> (a, *True, c) = (1, 2, 3)
Traceback (most recent call last):
SyntaxError: cannot assign to True

• assign to __debug__ (inside Tuple)

>>> (a, *__debug__, c) = (1, 2, 3)
Traceback (most recent call last):
SyntaxError: cannot assign to __debug__

• assign to operator

>>> [a, b, c + 1] = [1, 2, 3]
Traceback (most recent call last):
SyntaxError: cannot assign to operator

• assign to conditional expression

>>> a if 1 else b = 1
Traceback (most recent call last):
SyntaxError: cannot assign to conditional expression

• assign to None

# From compiler_complex_args():
>>> def f(None=1):
...     pass
Traceback (most recent call last):
SyntaxError: invalid syntax

• non-default argument follows default

>>> def f(x, y=1, z):
...     pass
Traceback (most recent call last):
SyntaxError: non-default argument follows default argument

• None as function parameter in definition

>>> def f(x, None):
...     pass
Traceback (most recent call last):
SyntaxError: invalid syntax

• None as function parameter in definition

>>> def f(*None):
...     pass
Traceback (most recent call last):
SyntaxError: invalid syntax

• None as function parameter in definition

>>> def f(**None):
...     pass
Traceback (most recent call last):
SyntaxError: invalid syntax

• None as function name

>>> def None(x):
...     pass
Traceback (most recent call last):
SyntaxError: invalid syntax

• Generator expression must be parenthesized

>>> def f(it, *varargs, **kwargs):
...     return list(it)
>>> L = range(10)
>>> f(x for x in L)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> f(x for x in L, 1)
Traceback (most recent call last):
SyntaxError: Generator expression must be parenthesized
>>> f(x for x in L, y=1)
Traceback (most recent call last):
SyntaxError: Generator expression must be parenthesized
>>> f(x for x in L, *[])
Traceback (most recent call last):
SyntaxError: Generator expression must be parenthesized
>>> f(x for x in L, **{})
Traceback (most recent call last):
SyntaxError: Generator expression must be parenthesized
>>> f(L, x for x in L)
Traceback (most recent call last):
SyntaxError: Generator expression must be parenthesized
>>> f(x for x in L, y for y in L)
Traceback (most recent call last):
SyntaxError: Generator expression must be parenthesized
>>> f(x for x in L,)
Traceback (most recent call last):
SyntaxError: Generator expression must be parenthesized
>>> f((x for x in L), 1)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> class C(x for x in L):
...     pass
Traceback (most recent call last):
SyntaxError: invalid syntax

• perhaps meant ==

>>> f(lambda x: x[0] = 3)
Traceback (most recent call last):
SyntaxError: expression cannot contain assignment, perhaps you meant "=="?
>>> f(x()=2)
Traceback (most recent call last):
SyntaxError: expression cannot contain assignment, perhaps you meant "=="?
>>> f(a or b=1)
Traceback (most recent call last):
SyntaxError: expression cannot contain assignment, perhaps you meant "=="?
>>> f(x.y=1)
Traceback (most recent call last):
SyntaxError: expression cannot contain assignment, perhaps you meant "=="?
>>> f((x)=2)
Traceback (most recent call last):
SyntaxError: expression cannot contain assignment, perhaps you meant "=="?

• assign to True

>>> f(True=2)
Traceback (most recent call last):
SyntaxError: cannot assign to True

• assign to __debug__

>>> f(__debug__=1)
Traceback (most recent call last):
SyntaxError: cannot assign to __debug__

• assign to generator expression

>>> (x for x in x) += 1
Traceback (most recent call last):
SyntaxError: cannot assign to generator expression

• assign to None

>>> None += 1
Traceback (most recent call last):
SyntaxError: cannot assign to None

• assign to __debug__

>>> __debug__ += 1
Traceback (most recent call last):
SyntaxError: cannot assign to __debug__

• assign to function call

>>> f() += 1
Traceback (most recent call last):
SyntaxError: cannot assign to function call

• continue not properly in loop

    >>> def foo():
    ...     try:
    ...         pass
    ...     finally:
    ...         continue
    Traceback (most recent call last):
      ...
    SyntaxError: 'continue' not properly in loop

• break outside loop

   >>> try:
   ...     print(1)
   ...     break
   ...     print(2)
   ... finally:
   ...     print(3)
   Traceback (most recent call last):
     ...
   SyntaxError: 'break' outside loop

• too many statically nested blocks; will not include unless reported by an actual user

   >>> while 1:
   ...  while 2:
   ...   while 3:
   ...    while 4:
   ...     while 5:
   ...      while 6:
   ...       while 8:
   ...        while 9:
   ...         while 10:
   ...          while 11:
   ...           while 12:
   ...            while 13:
   ...             while 14:
   ...              while 15:
   ...               while 16:
   ...                while 17:
   ...                 while 18:
   ...                  while 19:
   ...                   while 20:
   ...                    while 21:
   ...                     while 22:
   ...                      break
   Traceback (most recent call last):
     ...
   SyntaxError: too many statically nested blocks

• name use prior to global declaration

   >>> def f():
   ...     print(x)
   ...     global x
   Traceback (most recent call last):
     ...
   SyntaxError: name 'x' is used prior to global declaration

• name assigned to before global declaration

   >>> def f():
   ...     x = 1
   ...     global x
   Traceback (most recent call last):
     ...
   SyntaxError: name 'x' is assigned to before global declaration

• name is parameter and global

   >>> def f(x):
   ...     global x
   Traceback (most recent call last):
     ...
   SyntaxError: name 'x' is parameter and global

• name used prior to nonlocal declaration

   >>> def f():
   ...     x = 1
   ...     def g():
   ...         print(x)
   ...         nonlocal x
   Traceback (most recent call last):
     ...
   SyntaxError: name 'x' is used prior to nonlocal declaration

• name is assigned to before nonlocal declaration

   >>> def f():
   ...     x = 1
   ...     def g():
   ...         x = 2
   ...         nonlocal x
   Traceback (most recent call last):
     ...
   SyntaxError: name 'x' is assigned to before nonlocal declaration

• name is parameter and nonlocal

   >>> def f(x):
   ...     nonlocal x
   Traceback (most recent call last):
     ...
   SyntaxError: name 'x' is parameter and nonlocal

• name is nonlocal and global

   >>> def f():
   ...     global x
   ...     nonlocal x
   Traceback (most recent call last):
     ...
   SyntaxError: name 'x' is nonlocal and global

• no binding for nonlocal found

   >>> def f():
   ...     nonlocal x
   Traceback (most recent call last):
     ...
   SyntaxError: no binding for nonlocal 'x' found

• nonlocal not allowed at module level

From SF bug #1705365
   >>> nonlocal x
   Traceback (most recent call last):
     ...
   SyntaxError: nonlocal declaration not allowed at module level

• no binding for nonlocal found

From https://bugs.python.org/issue25973
   >>> class A:
   ...     def f(self):
   ...         nonlocal __x
   Traceback (most recent call last):
     ...
   SyntaxError: no binding for nonlocal '_A__x' found

• cannot assign to function call

   >>> if 1:
   ...   x() = 1
   ... elif 1:
   ...   pass
   Traceback (most recent call last):
     ...
   SyntaxError: cannot assign to function call

• raise single exception

Make sure that the old "raise X, Y[, Z]" form is gone:
   >>> raise X, Y
   Traceback (most recent call last):
     ...
   SyntaxError: invalid syntax
   >>> raise X, Y, Z
   Traceback (most recent call last):
     ...
   SyntaxError: invalid syntax

• keyword argument repeated

>>> f(a=23, a=234)
Traceback (most recent call last):
   ...
SyntaxError: keyword argument repeated

• cannot assign to set display

>>> {1, 2, 3} = 42
Traceback (most recent call last):
SyntaxError: cannot assign to set display

• cannot assign to dictdisplay

>>> {1: 2, 3: 4} = 42
Traceback (most recent call last):
SyntaxError: cannot assign to dict display

• cannot assign to f-string expression

>>> f'{x}' = 42
Traceback (most recent call last):
SyntaxError: cannot assign to f-string expression

• named arguments must follow bare * # ignoring this for now

    >>> with (lambda *:0): pass
    Traceback (most recent call last):
    SyntaxError: named arguments must follow bare *

I see a lot of:

Traceback (most recent call last):
  File "solution.py", line 1
    >>>print("Hello World!")
     ^
SyntaxError: invalid syntax

from newcomers code, is this the kind of stuff friendly-traceback also want to spot and write a nice message like: "Looks like you copy-pasted from an interactive interpreter, you also took the >>> prompt, which is only presentation in the interpreter, not actual Python code".

If this is also the kind of stuff friendly-traceback want to catch, I'll gladly implement it myself.

Creating an issue so that anyone else does not duplicate my work.

I'm experimenting with public API methods (eg. explain). My IDE (PyDev) doesn't see them attached to friendly_traceback module. Probably it's because they are missing from friendly_traceback.public_api.__all__ list.

Currently, the development of Friendly-traceback is done using Python 3.7. However, error messages in older versions of Python are, in some instances, different. This needs to be taken into account, and a decision made as to how many different versions of Python we need to support.

For now, I plan to support 3.6 and 3.7 (and 3.8 when a stable version is released). I'm leaving this issue open for discussion.

= = =
Edit: I've installed 3.8.0a and found one small change in a SyntaxError message.

Hello,

As requested, I am reporting an issue in which an exception was encountered but no information was known. This was in version 0.0.28 running in Thonny 3.2.7.

Code to repro:

class Vehicle:
    '''Vehicle parent class with class and instance variables'''
    wheels = 4

    def __init__(self, make, model):
        self.make = make
        self.model = model

class Car(Vehicle):
    '''Car child class'''
    
    def __init__(self, color, mileage):
        self.color = color
        self.mileage = mileage

my_car = Car('Green', 300)
my_car.model

Error:

Python exception:
AttributeError: 'Car' object has no attribute 'model'

No information is known about this exception.
Please report this example to
https://github.com/aroberge/friendly-traceback/issues

Execution stopped on line 1 of file '<pyshell>'.

Problem: source of '' is not available

Thanks and keep up the good work!

To facilitate the integration into other projects, I want to create a demo where I do so, and possibly document the steps taken in a series of blog posts. After a few experiments, I have decided to use IDLE as a base editor and use monkeypatching to incorporate friendly-traceback.

• The invocation will be done using --idle as a command line option

IDLE itself has many command line options. This demonstration should support the following

• by default, it should open a shell window (-i in IDLE)
• if -e is specified (together with --idle), it should open an editor instead [same flag as IDLE]
• if a source file is specified (together with --idle), it should run this file in IDLE's shell [similar to -r file in IDLE]

Some additional features

• Ability to change the language via a menu in IDLE
• The "friendly" output shoud be shown in a separate window, with custom formatting.

SyntaxError : with its often terse description given by Python (SyntaxError: invalid syntax), they are often the most confusing exceptions encountered by beginners. Multiple, and very different causes can give rise to these exceptions. In an ideal situation, friendly-traceback should be able to analyze the code, and provide some specific and accurate explanation, such as

• you forgot to include a colon : at the end of your if (or other) statement on line xx
• you forgot to include parentheses () when you defined your function on line xx
• you likely forgot to have a space between def and your function name as you were attempting to define a new function
• etc.

It might be helpful to explore different methods to perform some analysis of SyntaxErrors, with each method selectable by a different option when running friendly-traceback. Ideally, friendly-traceback should not have external dependencies; however, this might not be the best strategy. Perhaps, one could use tools like pylint or pyflakes to analyze the code and use the output of these tools to give the information needed. However, it should be noted that, sometimes, the code being evaluated will not be saved in a file, but will be in strings (when using an REPL).

While I intend to explore ways of doing this on my own, I would welcome some help on this task, either simply by contributing to the discussion on this issue, or by people wishing to explore different solutions.

An extremely common case I see beginners trip up on is an unclosed bracket, such as:

x = int('1'
if x == 1:
  print('yippee')

This is the worst case of this type, as the CPython parser will start and fail to parse a ternary expression, pointing to the colon! The same issue arises with for loops (where they would begin a generator expression or comprehension).

  File "/tmp/test.py", line 2
    if a == 1:
             ^
SyntaxError: invalid syntax

Other tokens are slightly less bad here, where the error will point to the end of the first token:

x = int('1'
while x == 1:
  ...

  File "/tmp/test3.py", line 2
    while a == 1:
        ^
SyntaxError: invalid syntax

PyPy has a significantly nicer error message for this entire scenario:

  File "/tmp/test.py", line 1
    a = int('1'
           ^
SyntaxError: parenthesis is never closed

Hello @aroberge!
The blog post linked from a recent PSF update links to a missing page: in “Glance through of exceptions on this page”, the URL is missing a version like _3.8. Same for the following link, "included here".

(I hope this is an OK-ish place to report this. The blog seems to have a comment mechanism but my ad blocker doesn't trust it.)

The following:

python -m friendly_traceback test

    Python exception:
        FileNotFoundError: [Errno 2] No such file or directory: 'test'

    A FileNotFoundError exception indicates that you
    are trying to open a file that cannot be found by Python.
    This could be because you misspelled the name of the file.

    Likely cause based on the information given by Python:
                I suspect that you are using a regular
                Python console after importing Friendly-traceback.
                Unfortunately, no further processing can be done.

gives an incorrect "Likely cause". We should be able to intercept this and give a better information message.

For example, instead of

The opening square bracket '[' on line 5 is not closed.

        5:     return [1, 2, 3

Show

The opening square bracket '[' on line 5 is not closed.

        5:     return [1, 2, 3
                      ^

as this might remove some ambuiguity.

To integrate with AvantPy, error messages need to support keyword translations.

For example, if one attempts to type

print "hello"

an error message is generated, indicating that print is now a function. However, using AvantPy, the original source might not contain print, but perhaps something like imprimer instead. Support for showing both variants, as in imprimer/print should be included.

The more exceptions will be included in friendly-traceback, the more useful it will be.
As you can see below, there is a rather large number of Exceptions that can be raised in Python.
If you feel like adding coverage for one exception, you are more than welcome to help.

Have a look at https://aroberge.github.io/friendly-traceback-docs/docs/html/adding_exception.html
to see what this requires. In the following, the few exceptions whose name is followed by a * are included; the rest is up for grab, one exception at a time.

Some exceptions to this: SystemExit and KeyboardInterrupt should probably not be included. friendly-traceback is meant to catch exceptions and provide an explanation for them. When either is raised, this is likely not what should happen. GeneratorExit, StopIteration, and StopAsyncIteration are excluded as they should not normally be seen by an end user - at least, not by beginners who would need
additional explanation about the meaning of such exceptions. Furthermore, in the case of
StopIteration, see https://www.python.org/dev/peps/pep-0479/
FloatingPointError is not used by Python (https://docs.python.org/3.7/library/exceptions.html#FloatingPointError).
ArithmeticError is a base class for other exceptions; it is not normally seen; similarly for BaseException and Exception.

Some base class exceptions, which are never directly raised by Python, should probably be handled differently.

    BaseException **
     +-- SystemExit **
     +-- KeyboardInterrupt **
     +-- GeneratorExit **
     +-- Exception **
          +-- StopIteration **
          +-- StopAsyncIteration **
          +-- ArithmeticError **
          |    +-- FloatingPointError **
          |    +-- OverflowError
          |    +-- ZeroDivisionError *
          +-- AssertionError
          +-- AttributeError
          +-- BufferError
          +-- EOFError
          +-- ImportError *
          |    +-- ModuleNotFoundError *
          +-- LookupError *
          |    +-- IndexError *
          |    +-- KeyError *
          +-- MemoryError
          +-- NameError  *
          |    +-- UnboundLocalError *
          +-- OSError
          |    +-- BlockingIOError
          |    +-- ChildProcessError
          |    +-- ConnectionError
          |    |    +-- BrokenPipeError
          |    |    +-- ConnectionAbortedError
          |    |    +-- ConnectionRefusedError
          |    |    +-- ConnectionResetError
          |    +-- FileExistsError
          |    +-- FileNotFoundError
          |    +-- InterruptedError
          |    +-- IsADirectoryError
          |    +-- NotADirectoryError
          |    +-- PermissionError
          |    +-- ProcessLookupError
          |    +-- TimeoutError
          +-- ReferenceError
          +-- RuntimeError
          |    +-- NotImplementedError
          |    +-- RecursionError
          +-- SyntaxError *
          |    +-- IndentationError *
          |         +-- TabError *
          +-- SystemError
          +-- TypeError
          +-- ValueError
          |    +-- UnicodeError
          |         +-- UnicodeDecodeError
          |         +-- UnicodeEncodeError
          |         +-- UnicodeTranslateError
          +-- Warning
               +-- DeprecationWarning
               +-- PendingDeprecationWarning
               +-- RuntimeWarning
               +-- SyntaxWarning
               +-- UserWarning
               +-- FutureWarning
               +-- ImportWarning
               +-- UnicodeWarning
               +-- BytesWarning
               +-- ResourceWarning

Is there any chance of a debian package, e.g. python3-friendly-traceback ?

This would make it easier to use friendly-traceback in python/gtk based projects there.

>>> word = "Hello"
>>> print(len.(word))

    Python exception:
        SyntaxError: invalid syntax

    A SyntaxError occurs when Python cannot understand your code.

    Python could not understand the code in the file
    '<friendly-console:5>'
    beyond the location indicated below by --> and ^.

    -->1: print(len.(word))
                    ^

    I don't have enough information from Python:
        I make an effort below to guess what caused the problem
        but I might guess incorrectly.

        It is possible that you forgot a comma between items in a tuple,
        or between function arguments,
        before the position indicated by --> and ^.

I cannot think of any situation where a dot preceding a paren, or any other type of bracket, would be valid. A more specific error message would definitely be useful here.