If we could include column offsets from the AST nodes for every bytecode instructions we could leverage these to offer much better tracebacks and a lot more information to debuggers and similar tools. For instance, in an expression such as:

z['aaaaaa']['bbbbbb']['cccccc']['dddddd].sddfsdf.sdfsdf

where one of these elements is None, we could tell exactly what of these pieces is the actual None and we could do some cool highlighting on tracebacks.

Similarly, coverage tools and debuggers could also make this distinction and therefore could offer reports with more granularity.

The cost is not 0: it would be two integers per bytecode instruction, but I think it may be worth the effort.

I'm going to prepare a PEP since the discussion regarding if the two integers per bytecode are worth enough is going to be eternal.

The additional cost will not only be the line number table, but we need to store the line for exceptions that are reraised after cleanup.
Adding a column will mean more stack consumption.

Yup, but I still think is worth the cost, giving that debugging improvements are usually extremely popular among users.

Specific examples of current messages and proposed improvements would help focus discussion.

If you are willing to only handle code lines up to 256 chars, only 2 bytes should be needed.  (0,0) or (255,255) could mean 'somewhere beyond the 256th char'.

> Specific examples of current messages and proposed improvements would help focus discussion.


Yeah, I am proposing going from:

>>> x['aaaaaa']['bbbbbb']['cccccc']['dddddd'].sddfsdf.sdfsdf
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'NoneType' object is not subscriptable

to

>>> x['aaaaaa']['bbbbbb']['cccccc']['dddddd'].sddfsdf.sdfsdf
                         ^^^^^^^^^^
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'NoneType' object is not subscriptable

Basically, to highlight in all exceptions the range in the displayed line where the error ocurred. For instance:


>>> foo(a, b/z+2, c, 132432 /x, d /y)
                     ^^^^^^^^^
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ZeroDivisionError: division by zero

Marking what expression evaluated to None would be extremely helpful.  So would marking the 0 denominator when there is more than one candidate: "e = a/b + c/d".  It should be easy to revise IDLE Shell's print_exception to tag the span.  In some cases, the code that goes from a traceback line to a line in a file and marks it could do the same.

What would you do when the expression is not the last line?

try:
    x/y
    ...
except Exception as e:
   ...
   raise e

The except and raise might even be in a separate module.
I look forward to the PEP and discussion.

> So would marking the 0 denominator when there is more than one candidate: "e = a/b + c/d".

No, it will mark the offset of the bytecode that was getting executed when the exception was raised. Is just a way to mark what is raising the particular exception

> What would you do when the expression is not the last line?

There is some logic needed to re-raise exceptions. But basically it boils down to comparate the line numbers to decide if you want to propagate or not the offsets.

New changeset 98eee94421dcb42c15f2d7fc4cd21357722fbe2a by Pablo Galindo in branch 'main':
bpo-43950: Add code.co_positions (PEP 657) (GH-26955)
https://github.com/python/cpython/commit/98eee94421dcb42c15f2d7fc4cd21357722fbe2a

New changeset ec8759b060eff83ff466f42c5a96d83a685016ce by Batuhan Taskaya in branch 'main':
bpo-43950: optimize column table assembling with pre-sizing object (GH-26997)
https://github.com/python/cpython/commit/ec8759b060eff83ff466f42c5a96d83a685016ce

New changeset 44f91fc802a2b71312e9abd9e8e9dcbf02e8216d by Batuhan Taskaya in branch 'main':
bpo-43950: use 0-indexed column offsets for bytecode positions (GH-27011)
https://github.com/python/cpython/commit/44f91fc802a2b71312e9abd9e8e9dcbf02e8216d

New changeset 693cec0e2dcafa393ed5cdaa606f8dc8e3876adf by Batuhan Taskaya in branch 'main':
bpo-43950: include position in dis.Instruction (GH-27015)
https://github.com/python/cpython/commit/693cec0e2dcafa393ed5cdaa606f8dc8e3876adf

New changeset 5644c7b3ffd49bed58dc095be6e6148e0bb4431e by Ammar Askar in branch 'main':
bpo-43950: Print columns in tracebacks (PEP 657) (GH-26958)
https://github.com/python/cpython/commit/5644c7b3ffd49bed58dc095be6e6148e0bb4431e

New changeset 4823d9a51281ebbc8e8d82a0dd3edc7d13ea8ac7 by Ammar Askar in branch 'main':
bpo-43950: Add option to opt-out of PEP-657 (GH-27023)
https://github.com/python/cpython/commit/4823d9a51281ebbc8e8d82a0dd3edc7d13ea8ac7

New changeset 1890dd235f618d60c938f6904d2e1a8a56f99c1c by Batuhan Taskaya in branch 'main':
bpo-43950: Specialize tracebacks for subscripts/binary ops (GH-27037)
https://github.com/python/cpython/commit/1890dd235f618d60c938f6904d2e1a8a56f99c1c

New changeset 9c3eaf88dc5d5bed80cc45936de06b7b3162bc6d by Ammar Askar in branch 'main':
bpo-43950: Add documentation for PEP-657 (GH-27047)
https://github.com/python/cpython/commit/9c3eaf88dc5d5bed80cc45936de06b7b3162bc6d

New changeset 919ad537510fdc2c750109e0bc4eceea234324b2 by Batuhan Taskaya in branch 'main':
bpo-43950: make BinOp specializations more reliable (GH-27126)
https://github.com/python/cpython/commit/919ad537510fdc2c750109e0bc4eceea234324b2

As reported by Will McGugan on twitter:

Printing may be wrong for some unicode characters like:


Traceback (most recent call last):
  File "test.py", line 1, in <module>
    x = ("该" / 0) + (1 / 2)
         ~~~~~~^
TypeError: unsupported operand type(s) for /: 'str' and 'int'


and


Traceback (most recent call last):
  File "test.py", line 3, in <module>
    x = ("💩" / 0) + (1 / 2)
         ~~~~~~~
TypeError: unsupported operand type(s) for /: 'str' and 'int'