Skip to content

TST: Adjust tests and require_geos decorator implementation for docstring indent stripping in Python 3.13#1938

Merged
jorisvandenbossche merged 3 commits intoshapely:mainfrom
musicinmybrain:python3.13
Feb 16, 2024
Merged

TST: Adjust tests and require_geos decorator implementation for docstring indent stripping in Python 3.13#1938
jorisvandenbossche merged 3 commits intoshapely:mainfrom
musicinmybrain:python3.13

Conversation

@musicinmybrain
Copy link
Copy Markdown
Contributor

Fixes #1937.

I’m not thrilled with this, particularly with the unexplained extra_indent, but it does successfully match the behavior in Python 3.13.0a1.

For reference, here’s the current implementation of _PyCompile_CleanDoc():

https://github.com/methane/cpython/blob/9c03215a3ee31462045b2c2ee162bdda30c54572/Python/compile.c#L7816

@coveralls
Copy link
Copy Markdown

coveralls commented Nov 22, 2023

Pull Request Test Coverage Report for Build 7929712003

Details

  • 0 of 1 (100.0%) changed or added relevant line in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall first build on python3.13 at 88.076%

Totals Coverage Status
Change from base Build 7929487664: 88.1%
Covered Lines: 2600
Relevant Lines: 2952

💛 - Coveralls

@jorisvandenbossche jorisvandenbossche changed the title Adjust tests for docstring indent stripping in Python 3.13 TST: Adjust tests for docstring indent stripping in Python 3.13 Nov 26, 2023
@jorisvandenbossche
Copy link
Copy Markdown
Member

@musicinmybrain Thanks for looking into this! We don't yet have CI for Python 3.13, but if you can test it on your end that this passes, I think we are certainly happy to already merge it.

particularly with the unexplained extra_indent

That's maybe a bug to report to cpython? If I try with one of our docstrings with a .. note:: using Python 3.11, that additional indent isn't present.

@musicinmybrain
Copy link
Copy Markdown
Contributor Author

I tried originally to form a simplified reproducer. For example, if I write foo.py:

class SomeClass:
    def func(self):
        """Docstring that will be mocked.
        A multiline.

        .. note:: 'func' requires at least GEOS {version}.

        Some description.
        """

and then do

python3.13 -c 'from foo import SomeClass; print(repr(SomeClass.func.__doc__))'

I get what I would expect,

"Docstring that will be mocked.\nA multiline.\n\n.. note:: 'func' requires at least GEOS {version}.\n\nSome description.\n"

Now that I’m returning to this after a few days, I see exactly what’s happening. The docstring modification algorithm runs on the introspected (i.e. after byte-compilation) docstring, and it assumes the indentation level is at least 2. Since the doc string has already had indents stripped, this isn’t a good assumption, but we end up with the note indented two spaces anyway.

# Insert the message at the first double newline
position = doc.find("\n\n") + 2
# Figure out the indentation level
indent = 2
while True:
if doc[position + indent] == " ":
indent += 1
else:
break
wrapped.__doc__ = doc.replace(
"\n\n", "\n\n{}.. note:: {}\n\n".format(" " * indent, msg), 1
)
.

It seems like the choice of 2 is just a tiny performance optimization, and we could get rid of the “extra indent” without breaking anything else by simply starting at 0. I’ll try it out.

@musicinmybrain
Copy link
Copy Markdown
Contributor Author

I began by force-pushing with an improved comment reflecting the above analysis.

This fixes “extra” indentation before the GEOS version note in Python
3.13, where the compiler strips indentation from docstrings.
@musicinmybrain
Copy link
Copy Markdown
Contributor Author

It seems like the choice of 2 is just a tiny performance optimization, and we could get rid of the “extra indent” without breaking anything else by simply starting at 0. I’ll try it out.

This approach appears to work as expected in my testing on Python 3.13 and 3.12. Updating this PR.

@musicinmybrain musicinmybrain changed the title TST: Adjust tests for docstring indent stripping in Python 3.13 TST: Adjust tests and require_geos decorator implementation for docstring indent stripping in Python 3.13 Nov 26, 2023
@musicinmybrain
Copy link
Copy Markdown
Contributor Author

If you wanted to find the indent more concisely without an explicit Python loop (in exchange for adding an otherwise-unnecessary temporary copy of part of the docstring), you could instead write:

# Figure out the indentation level
indent = len(doc) - len(doc[position:].lstrip(" ")) - position

@jorisvandenbossche
Copy link
Copy Markdown
Member

Sorry for the slow follow-up, but rebased and still passing, so going to merge this now (and include in shapely 2.0.3). Thanks again!

@jorisvandenbossche jorisvandenbossche merged commit 2fd1487 into shapely:main Feb 16, 2024
jorisvandenbossche pushed a commit to jorisvandenbossche/shapely that referenced this pull request Feb 16, 2024
jorisvandenbossche pushed a commit that referenced this pull request Feb 16, 2024
@theroggy theroggy added this to the 2.1 milestone Feb 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Test failure on Python 3.13 because it strips indents from docstrings

4 participants