This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: f_lasti behaves differently for lambdas returned from loops
Type: Stage:
Components: Versions: Python 3.11
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Mark.Shannon, nedbat
Priority: normal Keywords: 3.11regression

Created on 2022-01-02 13:45 by nedbat, last changed 2022-04-11 14:59 by admin.

Messages (4)
msg409496 - (view) Author: Ned Batchelder (nedbat) * (Python triager) Date: 2022-01-02 13:45
Python 3.11.0a3 changed how f_lasti behaves when calling a function in some
situations.

This example shows returning a lambda from a function.  If the lambda is
returned in a loop, then frame.f_lasti is 0 when it is called.  If the lambda
is not in a loop, then f_lasti is -1, as is usual when calling a function.

This behavior first appeared in 3.11.0a3.

---< showit.py >--------------------------------------------------

import linecache, sys

def trace(frame, event, arg):
    # The weird globals here is to avoid a NameError on shutdown...
    if frame.f_code.co_filename == globals().get("__file__"):
        if event != "line":
            lineno = frame.f_lineno
            lasti = f"lasti {frame.f_lasti}" if event == "call" else ""
            line = linecache.getline(__file__, lineno).rstrip()
            print("{} {} {:10}: {}".format(event[:4], lineno, lasti, line))
    return trace

print(sys.version)
sys.settrace(trace)

def run1():
    for i in range(1):
        return lambda: i

def run2():
    return lambda: 99

f = run1()
print(f())

f = run2()
print(f())
------------------------------------------------------------------

Here are 3.10, 3.11.0a2, 3.11.0a3, and the tip of 3.11.  The changing line is
marked with <<<<<:

$ python3.10 showit.py
3.10.1 (main, Dec 14 2021, 08:30:13) [Clang 12.0.0 (clang-1200.0.32.29)]
call 16 lasti -1  : def run1():
retu 18           :         return lambda: i
call 18 lasti -1  :         return lambda: i        <<<<<
retu 18           :         return lambda: i
0
call 20 lasti -1  : def run2():
retu 21           :     return lambda: 99
call 21 lasti -1  :     return lambda: 99
retu 21           :     return lambda: 99
99

$ /usr/local/pyenv/pyenv/versions/3.11.0a2/bin/python3.11 showit.py
3.11.0a2 (main, Nov  6 2021, 07:16:30) [Clang 12.0.0 (clang-1200.0.32.29)]
call 16 lasti -1  : def run1():
retu 18           :         return lambda: i
call 18 lasti -1  :         return lambda: i        <<<<<
retu 18           :         return lambda: i
0
call 20 lasti -1  : def run2():
retu 21           :     return lambda: 99
call 21 lasti -1  :     return lambda: 99
retu 21           :     return lambda: 99
99

$ /usr/local/pyenv/pyenv/versions/3.11.0a3/bin/python3.11 showit.py
3.11.0a3 (main, Dec  9 2021, 12:22:18) [Clang 12.0.0 (clang-1200.0.32.29)]
call 16 lasti -1  : def run1():
retu 18           :         return lambda: i
call 18 lasti 0   :         return lambda: i        <<<<<
retu 18           :         return lambda: i
0
call 20 lasti -1  : def run2():
retu 21           :     return lambda: 99
call 21 lasti -1  :     return lambda: 99
retu 21           :     return lambda: 99
99

$ /usr/local/cpython/bin/python3 showit.py
3.11.0a3+ (heads/main:a82baed0e9, Jan  2 2022, 08:12:01) [Clang 12.0.0 (clang-1200.0.32.29)]
call 16 lasti -1  : def run1():
retu 18           :         return lambda: i
call 18 lasti 0   :         return lambda: i        <<<<<
retu 18           :         return lambda: i
0
call 20 lasti -1  : def run2():
retu 21           :     return lambda: 99
call 21 lasti -1  :     return lambda: 99
retu 21           :     return lambda: 99
99
msg409497 - (view) Author: Ned Batchelder (nedbat) * (Python triager) Date: 2022-01-02 13:54
Also, this only happens with a Python trace function.  With a C trace function, pPyFrameObject->f_frame->f_lasti is -1.
msg409845 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2022-01-06 13:40
Are you using `f_lasti == -1` as a proxy for "is this an actual call or a resumption after a yield"?

Ultimately PEP 669 will provide all the information you could need to differentiate different events.

In the meantime...

We now make all "call" events explicit in the bytecode using the new RESUME instruction.

That means the following is true for *all* call events:
  f_lasti >= 0
  code.co_code[f_lasti*2] == opcode.opmap["RESUME"]

The oparg of the RESUME instruction describes the kind of "call" event.
Given `oparg = code.co_code[f_lasti*2+1]`, then
`oparg == 0` implies that the event is an actual call.

See https://docs.python.org/3.11/library/dis.html#opcode-RESUME


Does that help?
msg410152 - (view) Author: Ned Batchelder (nedbat) * (Python triager) Date: 2022-01-09 14:40
Thanks, it does help.

While updating coverage.py to use the RESUME information, I found another problem:  https://bugs.python.org/issue46314
History
Date User Action Args
2022-04-11 14:59:54adminsetgithub: 90383
2022-01-09 14:40:45nedbatsetmessages: + msg410152
2022-01-06 13:40:41Mark.Shannonsetmessages: + msg409845
2022-01-02 13:54:56nedbatsetmessages: + msg409497
2022-01-02 13:45:09nedbatcreate