gh-142560: bytearray: prevent UAF in search-like methods by exporting self buffer by fatelei · Pull Request #142938 · python/cpython

fatelei · 2025-12-18T14:32:00Z

bytearray: prevent UAF in search-like methods by exporting self buffer

Fix a heap use-after-free when bytearray search helpers captured the raw
buffer pointer before normalizing the “sub” argument. A crafted index
or buffer provider could clear/resize the same bytearray during argument
conversion, invalidating the saved pointer and leading to UAF.

Change:
• For bytearray methods find/rfind/index/rindex/count/startswith/endswith/
contains/split/rsplit, export a temporary Py_buffer on self and pass
view.buf/view.len to the Py_bytes* helpers, then release it. While the
export is live, resizing/clearing raises BufferError, preventing stale
pointer dereferences.

Tests:
• Add re-entrancy tests to Lib/test/test_bytes.py that verify BufferError is
raised when index clears the target during find/count/index/rfind/rindex.

This mirrors existing protection used in bytearray.join and removes the
re-entrancy hazard without changing public APIs.

Issue: Use-after-free in bytearray search methods via re-entrant __index__ #142560

…buffer

Objects/bytearrayobject.c

vstinner

LGTM. Tests are now covered all modified methods, and methods now only use a cheap ob_exports++ and ob_exports--.

Objects/bytearrayobject.c

kumaraditya303 · 2025-12-19T07:37:07Z

The removed asserts were redundant as the functions are marked with @critical_section, they are useful when object is passed to another function.

… by exporting buffer in bytearray (pythonGH-142938) (cherry picked from commit 220f0b1) Co-authored-by: wangxiaolei <fatelei@gmail.com>

bedevere-bot · 2025-12-19T08:24:48Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Debian root 3.x (tier-1) has failed when building commit 220f0b1.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/345/builds/12988) and take a look at the build logs.
Check if the failure is related to this commit (220f0b1) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/345/builds/12988

Failed tests:

test_profiling

Summary of the results of the build (if available):

==

Click to see traceback logs

remote: Enumerating objects: 12, done.        
remote: Counting objects:   8% (1/12)        
remote: Counting objects:  16% (2/12)        
remote: Counting objects:  25% (3/12)        
remote: Counting objects:  33% (4/12)        
remote: Counting objects:  41% (5/12)        
remote: Counting objects:  50% (6/12)        
remote: Counting objects:  58% (7/12)        
remote: Counting objects:  66% (8/12)        
remote: Counting objects:  75% (9/12)        
remote: Counting objects:  83% (10/12)        
remote: Counting objects:  91% (11/12)        
remote: Counting objects: 100% (12/12)        
remote: Counting objects: 100% (12/12), done.        
remote: Compressing objects:   9% (1/11)        
remote: Compressing objects:  18% (2/11)        
remote: Compressing objects:  27% (3/11)        
remote: Compressing objects:  36% (4/11)        
remote: Compressing objects:  45% (5/11)        
remote: Compressing objects:  54% (6/11)        
remote: Compressing objects:  63% (7/11)        
remote: Compressing objects:  72% (8/11)        
remote: Compressing objects:  81% (9/11)        
remote: Compressing objects:  90% (10/11)        
remote: Compressing objects: 100% (11/11)        
remote: Compressing objects: 100% (11/11), done.        
remote: Total 12 (delta 1), reused 2 (delta 1), pack-reused 0 (from 0)        
From https://github.com/python/cpython
 * branch                    main       -> FETCH_HEAD
Note: switching to '220f0b107776391201a399c54dd01692c36fcdf4'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 220f0b10777 gh-142560: prevent use-after-free in search-like methods by exporting buffer in bytearray (#142938)
Switched to and reset branch 'main'

configure: WARNING: no system libmpdec found; falling back to pure-Python version for the decimal module
configure: WARNING: pkg-config is missing. Some dependencies may not be detected correctly.

make: *** [Makefile:2499: buildbottest] Error 2

cmaloney · 2025-12-19T08:32:31Z

Buildbot failure unrelated to this change

vstinner · 2025-12-19T13:26:49Z

Should we backport this change to 3.13 and 3.14?

miss-islington-app · 2025-12-19T16:31:31Z

Thanks @fatelei for the PR, and @kumaraditya303 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

miss-islington-app · 2025-12-19T16:31:31Z

Thanks @fatelei for the PR, and @kumaraditya303 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

…orting buffer in bytearray (pythonGH-142938) (cherry picked from commit 220f0b1) Co-authored-by: wangxiaolei <fatelei@gmail.com>

miss-islington-app · 2025-12-19T16:31:39Z

Sorry, @fatelei and @kumaraditya303, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 220f0b107776391201a399c54dd01692c36fcdf4 3.13

bedevere-app · 2025-12-19T16:31:44Z

GH-142983 is a backport of this pull request to the 3.14 branch.

cmaloney · 2025-12-19T16:32:22Z

I think so / the previous iteration of this PR that was closed was; working on the 3.13 backport

… by exporting buffer in bytearray (pythonGH-142938) (cherry picked from commit 220f0b1) Co-authored-by: wangxiaolei <fatelei@gmail.com>

bedevere-app · 2025-12-19T16:58:16Z

GH-142986 is a backport of this pull request to the 3.13 branch.

…porting buffer in bytearray (GH-142938) (#142983) gh-142560: prevent use-after-free in search-like methods by exporting buffer in bytearray (GH-142938) (cherry picked from commit 220f0b1) Co-authored-by: wangxiaolei <fatelei@gmail.com>

bedevere-bot · 2025-12-19T18:34:03Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 FreeBSD Refleaks 3.14 (tier-3) has failed when building commit dbc7fd6.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1800/builds/824) and take a look at the build logs.
Check if the failure is related to this commit (dbc7fd6) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1800/builds/824

Failed tests:

test_httpservers

Failed subtests:

test_large_content_length_truncated - test.test_httpservers.CGIHTTPServerTestCase.test_large_content_length_truncated

Test leaking resources:

test_events: memory blocks
test_events: references

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/test/support/__init__.py", line 847, in gc_collect
    gc.collect()
ResourceWarning: unclosed <socket.socket fd=9, family=2, type=1, proto=6, laddr=('127.0.0.1', 44486), raddr=('127.0.0.1', 44487)>
Task was destroyed but it is pending!
task: <Task pending name='Task-2677' coro=<BaseSelectorEventLoop._accept_connection2() done, defined at /buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/asyncio/selector_events.py:217> wait_for=<Future pending cb=[Task.task_wakeup()]>>
Warning -- Unraisable exception
Exception ignored while calling deallocator <function _SelectorTransport.__del__ at 0x8422bfd10>:
Traceback (most recent call last):
  File "/buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/asyncio/selector_events.py", line 873, in __del__
    _warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
ResourceWarning: unclosed transport <_SelectorSocketTransport closing fd=9>
k


Traceback (most recent call last):
  File "/buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/test/support/__init__.py", line 847, in gc_collect
    gc.collect()
ResourceWarning: unclosed <socket.socket fd=10, family=2, type=1, proto=6, laddr=('127.0.0.1', 44656), raddr=('127.0.0.1', 44657)>
Task was destroyed but it is pending!
task: <Task pending name='Task-3613' coro=<BaseSelectorEventLoop._accept_connection2() done, defined at /buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/asyncio/selector_events.py:217> wait_for=<Future pending cb=[Task.task_wakeup()]>>
Warning -- Unraisable exception
Exception ignored while calling deallocator <function _SelectorTransport.__del__ at 0x8422bfd10>:
Traceback (most recent call last):
  File "/buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/asyncio/selector_events.py", line 873, in __del__
    _warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
ResourceWarning: unclosed transport <_SelectorSocketTransport closing fd=10>
k


Traceback (most recent call last):
  File "/buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/test/test_httpservers.py", line 1140, in test_large_content_length_truncated
    self.assertEqual(res.read(), b'Hello World' + self.linesep)
                     ~~~~~~~~^^
  File "/buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/http/client.py", line 497, in read
    s = self.fp.read()
  File "/buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/socket.py", line 725, in readinto
    return self._sock.recv_into(b)
           ~~~~~~~~~~~~~~~~~~~~^^^
ConnectionResetError: [Errno 54] Connection reset by peer


Traceback (most recent call last):
  File "/buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/test/support/__init__.py", line 847, in gc_collect
    gc.collect()
ResourceWarning: unclosed <socket.socket fd=9, family=2, type=1, proto=6, laddr=('127.0.0.1', 44740), raddr=('127.0.0.1', 44741)>
Task was destroyed but it is pending!
task: <Task pending name='Task-4077' coro=<BaseSelectorEventLoop._accept_connection2() done, defined at /buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/asyncio/selector_events.py:217> wait_for=<Future finished exception=ConnectionResetError()>>
Future exception was never retrieved
future: <Future finished exception=ConnectionResetError()>
Traceback (most recent call last):
  File "/buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/asyncio/sslproto.py", line 581, in _on_handshake_complete
    raise handshake_exc
ConnectionResetError
Warning -- Unraisable exception
Exception ignored while calling deallocator <function _SelectorTransport.__del__ at 0x8422bfd10>:
Traceback (most recent call last):
  File "/buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/asyncio/selector_events.py", line 873, in __del__
    _warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
ResourceWarning: unclosed transport <_SelectorSocketTransport closing fd=9>
k


Traceback (most recent call last):
  File "/buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/test/support/__init__.py", line 847, in gc_collect
    gc.collect()
ResourceWarning: unclosed <socket.socket fd=10, family=2, type=1, proto=6, laddr=('127.0.0.1', 41899), raddr=('127.0.0.1', 41900)>
Task was destroyed but it is pending!
task: <Task pending name='Task-1508' coro=<BaseSelectorEventLoop._accept_connection2() done, defined at /buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/asyncio/selector_events.py:217> wait_for=<Future pending cb=[Task.task_wakeup()]>>
Warning -- Unraisable exception
Exception ignored while calling deallocator <function _SelectorTransport.__del__ at 0x8439ffd10>:
Traceback (most recent call last):
  File "/buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/asyncio/selector_events.py", line 873, in __del__
    _warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
ResourceWarning: unclosed transport <_SelectorSocketTransport closing fd=10>
k


Traceback (most recent call last):
  File "/buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/test/support/__init__.py", line 847, in gc_collect
    gc.collect()
ResourceWarning: unclosed <socket.socket fd=10, family=2, type=1, proto=6, laddr=('127.0.0.1', 42826), raddr=('127.0.0.1', 42827)>
Task was destroyed but it is pending!
task: <Task pending name='Task-3623' coro=<BaseSelectorEventLoop._accept_connection2() done, defined at /buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/asyncio/selector_events.py:217> wait_for=<Future finished exception=ConnectionResetError()>>
Future exception was never retrieved
future: <Future finished exception=ConnectionResetError()>
Traceback (most recent call last):
  File "/buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/asyncio/sslproto.py", line 581, in _on_handshake_complete
    raise handshake_exc
ConnectionResetError
Warning -- Unraisable exception
Exception ignored while calling deallocator <function _SelectorTransport.__del__ at 0x8439ffd10>:
Traceback (most recent call last):
  File "/buildbot/buildarea/3.14.ware-freebsd.refleak/build/Lib/asyncio/selector_events.py", line 873, in __del__
    _warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
ResourceWarning: unclosed transport <_SelectorSocketTransport closing fd=10>
k

cmaloney · 2025-12-21T19:47:08Z

I just spent a bit of time trying to remove the critical section but found a complicating case: After a call to _canresize succeeds increments of ob_exports must be "held up" (or fail) until after the resizing operation is complete.

Currently that is done by the critical section. Only one operation which depends on ob_exports being 0 or increments it off 0 at a time can run. To remove the critical section would need a new way to keep that invariant.

…orting buffer in bytearray (python#142938)

vstinner · 2025-12-22T10:47:26Z

I just spent a bit of time trying to remove the critical section

I suppose that your comment is related to the 3.13 backport, #142986.

…porting buffer in bytearray (GH-142938) (GH-142986) (cherry picked from commit 220f0b1) Co-authored-by: wangxiaolei <fatelei@gmail.com>

fix: bytearray: prevent UAF in search-like methods by exporting self …

1bb7bc0

…buffer

bedevere-app bot added the awaiting review label Dec 18, 2025

bedevere-app bot mentioned this pull request Dec 18, 2025

Use-after-free in bytearray search methods via re-entrant __index__ #142560

Closed

fatelei added 4 commits December 18, 2025 22:34

fix: fix code regen

cf35869

fix: fix code regen

0525ac5

fix: fix code regen

eb0a8a4

fix: fix code regen

f2afe09

vstinner reviewed Dec 18, 2025

View reviewed changes

Objects/bytearrayobject.c Show resolved Hide resolved

fatelei added 2 commits December 18, 2025 22:49

chore: add comment for bytearray_contains

00f6a54

chore: add comment for split and rsplit

be06c26

vstinner approved these changes Dec 18, 2025

View reviewed changes

bedevere-app bot added awaiting merge and removed awaiting review labels Dec 18, 2025

cmaloney approved these changes Dec 19, 2025

View reviewed changes

kumaraditya303 reviewed Dec 19, 2025

View reviewed changes

Objects/bytearrayobject.c Outdated Show resolved Hide resolved

kumaraditya303 reviewed Dec 19, 2025

View reviewed changes

Objects/bytearrayobject.c Outdated Show resolved Hide resolved

kumaraditya303 added 2 commits December 19, 2025 13:06

Update Objects/bytearrayobject.c

7f3b72f

Update Objects/bytearrayobject.c

1905c43

kumaraditya303 approved these changes Dec 19, 2025

View reviewed changes

kumaraditya303 enabled auto-merge (squash) December 19, 2025 07:38

kumaraditya303 merged commit 220f0b1 into python:main Dec 19, 2025
46 checks passed

bedevere-app bot removed the awaiting merge label Dec 19, 2025

cmaloney added needs backport to 3.13 bugs and security fixes needs backport to 3.14 bugs and security fixes labels Dec 19, 2025

miss-islington-app bot assigned kumaraditya303 Dec 19, 2025

bedevere-app bot removed the needs backport to 3.14 bugs and security fixes label Dec 19, 2025

bedevere-app bot removed the needs backport to 3.13 bugs and security fixes label Dec 19, 2025

kumaraditya303 removed their assignment Dec 19, 2025

cocolato pushed a commit to cocolato/cpython that referenced this pull request Dec 22, 2025

pythongh-142560: prevent use-after-free in search-like methods by exp…

778b1d1

…orting buffer in bytearray (python#142938)

Uh oh!

Conversation

fatelei commented Dec 18, 2025 • edited by bedevere-app bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

vstinner left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

kumaraditya303 commented Dec 19, 2025

Uh oh!

Uh oh!

bedevere-bot commented Dec 19, 2025

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

cmaloney commented Dec 19, 2025

Uh oh!

vstinner commented Dec 19, 2025

Uh oh!

miss-islington-app bot commented Dec 19, 2025

Uh oh!

miss-islington-app bot commented Dec 19, 2025

Uh oh!

miss-islington-app bot commented Dec 19, 2025

Uh oh!

bedevere-app bot commented Dec 19, 2025

Uh oh!

cmaloney commented Dec 19, 2025

Uh oh!

bedevere-app bot commented Dec 19, 2025

Uh oh!

bedevere-bot commented Dec 19, 2025

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

cmaloney commented Dec 21, 2025

Uh oh!

vstinner commented Dec 22, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

fatelei commented Dec 18, 2025 •

edited by bedevere-app bot

Loading