Any idea of the impact on memory consumption?

Typically, tuples and strs are dominants about number of objects.

In case of str, I expect 50+% of the objects are len(s)<8. So I think impact is "+8 bytes for 50+% strings". (56bytes -> 64bytes, 12.5%). But no impact for strings which 8 <= len(s) < 16. (Note that there is trailing NUL bytes. 8-length string like __init__ is 64bytes before and after this patch).

In case of tuples, roughly +8 bytes for 1/2 of tuples. But please note I have removed 8bytes from GC Header in Python 3.8. When comparing to Python 3.7, -8 bytes for 1/2 of tuples.

In case of str, I expect 50+% of the objects are len(s)<8. So I think impact is "+8 bytes for 50+% strings". (56bytes -> 64bytes, 12.5%). But no impact for strings which 8 <= len(s) < 16. (Note that there is trailing NUL bytes. 8-length string like init is 64bytes before and after this patch).

Ok, but would it be possible to measure the "real" memory usage of a Python program? For example, run "./python -i -m test test_os" and then run:

>>> import os; res=os.system(f"grep VmPeak /proc/{os.getpid()}/status")
VmPeak:	  224828 kB

I'm curious to know if we can see this memory footprint increase for real, not just in theory :-)

Ok, but would it be possible to measure the "real" memory usage of a Python program?

$ ./python -c 'import django, sys; sys._debugmallocstats()'

master:

10 arenas * 262144 bytes/arena     =            2,621,440

# bytes in allocated blocks        =            2,277,200
# bytes in available blocks        =              145,016
33 unused pools * 4096 bytes       =              135,168
# bytes lost to pool headers       =               29,136
# bytes lost to quantization       =               34,920
# bytes lost to arena alignment    =                    0
Total                              =            2,621,440

align-16byte:

10 arenas * 262144 bytes/arena     =            2,621,440

# bytes in allocated blocks        =            2,370,592
# bytes in available blocks        =               87,552
25 unused pools * 4096 bytes       =              102,400
# bytes lost to pool headers       =               29,520
# bytes lost to quantization       =               31,376
# bytes lost to arena alignment    =                    0
Total                              =            2,621,440

>>> (2370592-2277200) / 2277200 * 100
4.101176883892499

About 4% increase in this case.

$ ./python-master -c 'import django, os; os.system(f"grep VmPeak /proc/{os.getpid()}/status")'
VmPeak:    15624 kB

$ ./python-align-16byte -c 'import django, os; os.system(f"grep VmPeak /proc/{os.getpid()}/status")'
VmPeak:    15624 kB

No visible impact about VmPeak.

Also, can you run the benchmark suite?
Ideally, both CPU numbers and memory consumption numbers (it allows you to track memory consumption).

Also, can you run the benchmark suite?
Ideally, both CPU numbers and memory consumption numbers (it allows you to track memory consumption).

I post the result on bpo issue.

Thanks @methane for the PR 🌮🎉.. I'm working now to backport this PR to: 3.7.
🐍🍒⛏🤖

GH-13318 is a backport of this pull request to the 3.7 branch.

Thanks @methane for the PR 🌮🎉.. I'm working now to backport this PR to: 2.7.
🐍🍒⛏🤖

Sorry, @methane, I could not cleanly backport this to 2.7 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker f0be4bbb9b3cee876249c23f2ae6f38f43fa7495 2.7

GH-13319 is a backport of this pull request to the 2.7 branch.