Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve the error message for assert and AssertionError by adding location information #105724

Open
sobolevn opened this issue Jun 13, 2023 · 14 comments
Assignees
Labels
interpreter-core (Objects, Python, Grammar, and Parser dirs) type-feature A feature request or enhancement

Comments

@sobolevn
Copy link
Member

sobolevn commented Jun 13, 2023

Feature or enhancement

Right now assert generate errors that are hard to read, mostly because they lack context.
Example:

>>> assert 1 == 0
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AssertionError

Pitch

Let's make them better:

>>> assert 1 == 0
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    assert 1 == 0
           ^^^^^^
AssertionError

This is a good starting point.

Later this can be enhanced to use more context.

Linked PRs

@sobolevn sobolevn added type-feature A feature request or enhancement interpreter-core (Objects, Python, Grammar, and Parser dirs) labels Jun 13, 2023
@sobolevn sobolevn self-assigned this Jun 13, 2023
@sunmy2019
Copy link
Member

You need to modify Lib/traceback.py and (maybe) make _format_syntax_error more general.

@markshannon
Copy link
Member

You need to modify Lib/traceback.py and (maybe) make _format_syntax_error more general.

I don't think that is necessary. Changing the location of the RAISE_VARARGS instruction in the assertion should be sufficient.

@heysujal
Copy link

Can I work on this issue ?

@sobolevn
Copy link
Member Author

@heysujal sorry, I am already working on it :(
Please, feel free to take any other free issue. If you want any help, please feel free to ping me.

@sobolevn
Copy link
Member Author

sobolevn commented Jun 14, 2023

@markshannon @Fidget-Spinner I might be missing something, but looks like assert already supports this. There are two possible scenarios:

  1. assert is used in a named file, then it works
  2. assert is used in a repl, then it does not work (obviously)

Examples, file called ex.py:

def some():
    b = None
    assert b

some()

Result:

» ./python.exe ex.py
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/ex.py", line 8, in <module>
    some()
  File "/Users/sobolev/Desktop/cpython/ex.py", line 5, in some
    assert b
           ^
AssertionError

The same happens for CPython tests, if I add some failing assert to test_typing:

./python.exe -m test -v test_typing
======================================================================
FAIL: test_basics (test.test_typing.LiteralStringTests.test_basics)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_typing.py", line 328, in test_basics
    assert False
           ^^^^^
AssertionError

----------------------------------------------------------------------

But, REPL does not work for obvious reason: it cannot allow to read source lines. Example (the same code as in the first example):

>>> def some():
...     b = None
...     assert b
... 
>>> some()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in some
AssertionError

So, I am not sure what the task is.

@markshannon
Copy link
Member

markshannon commented Jun 14, 2023

What version of Python are you testing against?
I'm not seeing that behavior and the code suggests that the location is the whole line, not the expression.
https://github.com/python/cpython/blob/main/Python/compile.c#L3893

@sobolevn
Copy link
Member Author

I am using the main version.

@sobolevn
Copy link
Member Author

@markshannon never mind, I was running my own fork with the changes I made 🤦

@markshannon
Copy link
Member

Good to know that your changes work 🙂

@markshannon
Copy link
Member

We've all done it. I hate to think how much time I've wasted debugging the wrong branch in the past.

@gpshead gpshead changed the title Improve the error message for assert and AssertionError Improve the error message for assert and AssertionError by adding location information Jun 14, 2023
sobolevn added a commit to sobolevn/cpython that referenced this issue Jun 20, 2023
sobolevn added a commit to sobolevn/cpython that referenced this issue Jun 20, 2023
@markshannon
Copy link
Member

We don't want to increase the cost of execution if the assertion passes, which limits what transformations we can apply.
We can improve a few cases, though.

Boolean operation.

assert cond1 and cond2 is compiled as:

if not cond1 or not cond2:
    raise AssertionError # location `cond1 and cond2`

We could compile it as:

if not cond1:
    raise AssertionError # location `cond1`
if not cond2:
    raise AssertionError # location `cond2`

which is a bit bulkier, but is as fast as it executes the same VM instructions when the assert passes.

Comparison of local variable to constant or local variable

assert var1 == var2
Since we can recompute var1 or var2 without side effects, we can embed the values in the error message:

if not var1 == var2:
    _msg = f"{var1} != {var2}"
    raise AssertionError(_msg) # location `var1 == var2`

This is complicated somewhat if str(var1) raises.
We can add an instrinsic to handle the odd cases.

if not var1 == var2:
    instrinsic_assert_eq(var1, var2)

@sobolevn
Copy link
Member Author

I can surely work on this after the initial PR is merged 👍

@hugovk
Copy link
Member

hugovk commented Nov 10, 2023

I can surely work on this after the initial PR is merged 👍

@sobolevn Still stuff to do here, or can we close this issue?

@sobolevn
Copy link
Member Author

There are still tasks to complete in this issue.
Thanks for your work on closing issues! 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
interpreter-core (Objects, Python, Grammar, and Parser dirs) type-feature A feature request or enhancement
Projects
None yet
Development

No branches or pull requests

5 participants