[fix] Crash when parsing an empty arbitrary expression with ``extract_node`` by Pierre-Sassoulas · Pull Request #2736 · pylint-dev/astroid

Pierre-Sassoulas · 2025-05-12T19:54:12Z

Type of Changes

	Type
✓	🐛 Bug fix

Description

To be fair I'm not exactly sure of what the transient function is. It seems internal. But the fix seems self evident.

astroid/astroid/builder.py

Lines 32 to 35 in 3636bc2

    
           # The name of the transient function that is used to 
        
           # wrap expressions to be extracted when calling 
        
           # extract_node. 
        
           _TRANSIENT_FUNCTION = "__"

astroid/astroid/builder.py

Lines 303 to 315 in 3636bc2

    
           def _extract_expressions(node: nodes.NodeNG) -> Iterator[nodes.NodeNG]: 
        
               """Find expressions in a call to _TRANSIENT_FUNCTION and extract them. 
        
               The function walks the AST recursively to search for expressions that 
        
               are wrapped into a call to _TRANSIENT_FUNCTION. If it finds such an 
        
               expression, it completely removes the function call node from the tree, 
        
               replacing it by the wrapped expression inside the parent. 
        
               :param node: An astroid node. 
        
               :type node:  astroid.bases.NodeNG 
        
               :yields: The sequence of wrapped expressions on the modified tree 
        
               expression can be found. 
        
               """

astroid/astroid/builder.py

Lines 324 to 328 in 3636bc2

    
           # Search for node in all _astng_fields (the fields checked when 
        
           # get_children is called) of its parent. Some of those fields may 
        
           # be lists or tuples, in which case the elements need to be checked. 
        
           # When we find it, replace it by real_expr, so that the AST looks 
        
           # like no call to _TRANSIENT_FUNCTION ever took place.

Closes #2734

DanielNoord

If CI approves I approve as well :)

Pierre-Sassoulas · 2025-05-12T19:56:51Z

Do you understand the transient function though ? 😄

DanielNoord · 2025-05-12T19:59:18Z

See b8dc84f

It seems to be a very bespoke way to do extraction of nodes in extract_node.

codecov · 2025-05-12T19:59:57Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.21%. Comparing base (3636bc2) to head (e401578).
⚠️ Report is 58 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2736   +/-   ##
=======================================
  Coverage   93.21%   93.21%           
=======================================
  Files          93       93           
  Lines       11072    11072           
=======================================
  Hits        10321    10321           
  Misses        751      751

Flag	Coverage Δ
linux	`93.08% <ø> (ø)`
pypy	`93.21% <ø> (ø)`
windows	`93.19% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
astroid/builder.py	`95.13% <ø> (ø)`

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Pierre-Sassoulas · 2025-05-12T20:03:03Z

Ha, thank you, here's the doc:

     To extract arbitrary expressions, surround them with the fake
     function call __(...). After parsing, the surrounded expression
     will be returned and the whole AST (accessible via the returned
     node's parent attribute) will look like the function call was
     never there in the first place.

     Examples:

       >>> a = __(1)

       The const node will be extracted.

       >>> def x(d=__(foo.bar)): pass

       The node containing the default argument will be extracted.

       >>> def foo(a, b):
       >>>   return 0 < __(len(a)) < b

       The node containing the function call 'len' will be extracted.

    If no statements or expressions are selected, the last toplevel
    statement will be returned.
    If the selected statement is a discard statement, (i.e. an expression
    turned into a statement), the wrapped expression is returned instead.

I'm going to update the changelog then. Learned something today.

DanielNoord · 2025-05-12T20:04:08Z

I'm going to update the changelog then. Learned something today.

Don't we all when we look at astroid 😄

ChangeLog

Pierre-Sassoulas · 2025-05-12T20:16:55Z

Yeah, great piece of software, feels awesome.

…_node`` (#2736) Closes #2734 (cherry picked from commit 59f36e7)

…_node`` (#2736) (#2737) Closes #2734 (cherry picked from commit 59f36e7) Co-authored-by: Pierre Sassoulas <pierre.sassoulas@gmail.com>

* [fix] Crash when parsing an empty arbitrary expression with ``extract_node`` (#2736) (#2737) Closes #2734 (cherry picked from commit 59f36e7) Co-authored-by: Pierre Sassoulas <pierre.sassoulas@gmail.com> * [fix] Prevent crash on slice decorator for 'six' decorated function (#2738) (#2740) Closes #2721 (cherry picked from commit 555a128) Co-authored-by: Pierre Sassoulas <pierre.sassoulas@gmail.com> * [Backport maintenance/3.3.x] Initial fixes for Python 3.14 (#2747) (#2748) (cherry picked from commit 43111bb) * Improve backport job permissions (#2750) (cherry picked from commit fe5bb6c) * Use custom Github App to authenticate backport job (#2751) (#2752) (cherry picked from commit a6f8d6a) Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> * Include subclasses of standard property classes as property decorators (#2735) * Include subclasses of standard property types as property decorators * Modify astroid.bases and tests.test_nodes to reflect that enum.property was added in Python 3.11, not 3.10 * Apply suggestions from code review Co-authored-by: Pierre Sassoulas <pierre.sassoulas@gmail.com> --------- Co-authored-by: Pierre Sassoulas <pierre.sassoulas@gmail.com> (cherry picked from commit 30128b7) * Bump astroid to 3.3.11, update changelog (#2777) --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: pylint-backport-bot[bot] <212256041+pylint-backport-bot[bot]@users.noreply.github.com> Co-authored-by: Mitch Harding <mitchell.harding@hpe.com>

[fix] Crash when parsing '__()' (the transient function without args)

3161433

Closes #2734

Pierre-Sassoulas added this to the 3.3.11 milestone May 12, 2025

Pierre-Sassoulas added Crash 💥 backport maintenance/3.3.x labels May 12, 2025

Pierre-Sassoulas linked an issue May 12, 2025 that may be closed by this pull request

IndexError when calling extract_node #2734

Closed

Pierre-Sassoulas changed the title ~~[fix] Crash when parsing '__()' (the transient function without args)~~ [fix] Crash when parsing __() (the transient function without args) May 12, 2025

DanielNoord previously approved these changes May 12, 2025

View reviewed changes

Pierre-Sassoulas changed the title ~~[fix] Crash when parsing __() (the transient function without args)~~ [fix] Crash when parsing an empty arbitrary exception with extract_node May 12, 2025

Pierre-Sassoulas commented May 12, 2025

View reviewed changes

ChangeLog Outdated Show resolved Hide resolved

Pierre-Sassoulas changed the title ~~[fix] Crash when parsing an empty arbitrary exception with extract_node~~ [fix] Crash when parsing an empty arbitrary expression with extract_node May 12, 2025

Update ChangeLog

e401578

Pierre-Sassoulas dismissed DanielNoord’s stale review via e401578 May 12, 2025 20:33

DanielNoord approved these changes May 12, 2025

View reviewed changes

Pierre-Sassoulas enabled auto-merge (squash) May 12, 2025 20:36

Pierre-Sassoulas merged commit 59f36e7 into main May 12, 2025
20 checks passed

Pierre-Sassoulas deleted the 2734-indexerror-when-calling-extract_node branch May 12, 2025 20:43

github-actions bot pushed a commit that referenced this pull request May 12, 2025

[fix] Crash when parsing an empty arbitrary expression with ``extract…

0e297eb

…_node`` (#2736) Closes #2734 (cherry picked from commit 59f36e7)

github-actions bot mentioned this pull request May 12, 2025

[Backport maintenance/3.3.x] [fix] Crash when parsing an empty arbitrary expression with extract_node #2737

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[fix] Crash when parsing an empty arbitrary expression with `extract_node`#2736

[fix] Crash when parsing an empty arbitrary expression with `extract_node`#2736
Pierre-Sassoulas merged 2 commits intomainfrom
2734-indexerror-when-calling-extract_node

Pierre-Sassoulas commented May 12, 2025

Uh oh!

DanielNoord left a comment

Uh oh!

Pierre-Sassoulas commented May 12, 2025

Uh oh!

DanielNoord commented May 12, 2025

Uh oh!

codecov bot commented May 12, 2025 •

edited

Loading

Uh oh!

Pierre-Sassoulas commented May 12, 2025

Uh oh!

DanielNoord commented May 12, 2025

Uh oh!

Uh oh!

Pierre-Sassoulas commented May 12, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	# The name of the transient function that is used to
	# wrap expressions to be extracted when calling
	# extract_node.
	_TRANSIENT_FUNCTION = "__"

	def _extract_expressions(node: nodes.NodeNG) -> Iterator[nodes.NodeNG]:
	"""Find expressions in a call to _TRANSIENT_FUNCTION and extract them.

	The function walks the AST recursively to search for expressions that
	are wrapped into a call to _TRANSIENT_FUNCTION. If it finds such an
	expression, it completely removes the function call node from the tree,
	replacing it by the wrapped expression inside the parent.

	:param node: An astroid node.
	:type node: astroid.bases.NodeNG
	:yields: The sequence of wrapped expressions on the modified tree
	expression can be found.
	"""

	# Search for node in all _astng_fields (the fields checked when
	# get_children is called) of its parent. Some of those fields may
	# be lists or tuples, in which case the elements need to be checked.
	# When we find it, replace it by real_expr, so that the AST looks
	# like no call to _TRANSIENT_FUNCTION ever took place.

Uh oh!

Conversation

Pierre-Sassoulas commented May 12, 2025

Type of Changes

Description

Uh oh!

DanielNoord left a comment

Choose a reason for hiding this comment

Uh oh!

Pierre-Sassoulas commented May 12, 2025

Uh oh!

DanielNoord commented May 12, 2025

Uh oh!

codecov bot commented May 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Pierre-Sassoulas commented May 12, 2025

Uh oh!

DanielNoord commented May 12, 2025

Uh oh!

Uh oh!

Pierre-Sassoulas commented May 12, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

codecov bot commented May 12, 2025 •

edited

Loading