Enable testing on Python 3.12 by musicinmybrain · Pull Request #400 · aws/aws-xray-sdk-python

musicinmybrain · 2023-08-08T12:54:26Z

Issue #, if available:

Description of changes:

Added an explicitly dependency on setuptools, since a test imports pkg_resources.

Added Python 3.12 versions of all the tox environments that supported Python 3.11.

Added Python 3.12 to GitHub unit testing workflow.

I did not add Python 3.12 to the trove classifiers, but I’m happy to add this if it’s wanted.

Additional information:

Testing this PR requires #399.

It may also be necessary to use a very recent version of pip:

python3.12 -m venv _e
. _e/bin/activate
pip install --upgrade pip tox
tox

Most tests pass. Some HTTP tests seem to fail or flake in all Python versions due to transient issues at https://httpbin.org/. At a glance, regressions specific to Python 3.12 appear to be due to certain dependencies not being ready.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

musicinmybrain · 2023-08-08T13:08:56Z

Compare to: #376

musicinmybrain · 2023-08-09T02:18:36Z

The motivation for this PR is:

I am adding aws-xray-sdk-python to Fedora Linux, which already has Python 3.12 in the development version (Rawhide) and will release Fedora 39 with Python 3.12 this fall.
Since test dependencies are generated from (Python-version-specific) tox environments, I needed to patch in Python 3.12 versions of the existing environments to generate the proper dependencies for the RPM build process.
Offering necessary changes upstream is good practice.

carolabadeer · 2023-08-17T20:44:26Z

+    py{37,38,39,310,311,312}: pytest-asyncio
+
+    ; For pkg_resources
+    py{37,38,39,310,311,312}: setuptools


I am still not very clear on why this dependency is needed. You mentioned it was added since a test imports pkg_resources, which test are you referring to?

https://github.com/aws/aws-xray-sdk-python/blob/2976b25750d04ebe6dc7d0a3e399444896e82cae/tests/test_local_sampling_benchmark.py

Specifically:

aws-xray-sdk-python/tests/test_local_sampling_benchmark.py

Line 3 in 2976b25

from pkg_resources import resource_filename

aws-xray-sdk-python/tests/test_local_sampling_benchmark.py

Line 15 in 2976b25

with open(resource_filename(__name__, 'mock_sampling_rule.json')) as f:

The pkg_resources module is deprecated, but this test appears to be intentionally and specifically verifying compatibility with it.

Probably only “necessary” on Python 3.12 though:

gh-95299: Do not pre-install setuptools in virtual environments created with venv. This means that distutils, setuptools, pkg_resources, and easy_install will no longer available by default; to access these run pip install setuptools in the activated virtual environment.

https://docs.python.org/3.12/whatsnew/3.12.html

Perhaps, but it’s not harmful on others. The dependency is still real; it’s just allowed to be implicit. I guess I can change to py312 only if implicit is better than explicit.

michael-k · 2023-10-06T11:42:05Z

      py39: 3.9
      py310: '3.10'
      py311: '3.11'
+      py312: '3.12'


FYI: Should have worked at the time with 3.12.0-beta - 3.12. But it's available now, see https://github.com/actions/python-versions/blob/main/versions-manifest.json

Maybe rebase to trigger a rerun?

Required by the core tests in tests/test_local_sampling_benchmark.py.

Co-authored-by: Carol Abadeer <60774943+carolabadeer@users.noreply.github.com>

musicinmybrain · 2023-10-06T12:04:16Z

Hmm, I’m not sure why this is failing on Python 3.12 with ModuleNotFoundError: No module named 'setuptools' from setup.py. In local testing with tox, the added dependency on setuptools in tox.ini works as expected.

michael-k · 2023-10-06T14:01:39Z

run: pip install "tox<=3.27.1" -U tox-factor should install setuptools as well. The added dependency is only available inside the venv that tox creates. But it's required outside as well to build the sdist.

aws-xray-sdk-python/.github/workflows/UnitTesting.yaml

Lines 42 to 43 in 8563afc

    
           - name: Install tox 
        
             run: pip install "tox<=3.27.1" -U tox-factor

Ensure it is available for creating the sdist on Python 3.12+. aws#400 (comment)

musicinmybrain · 2023-11-04T00:43:21Z

run: pip install "tox<=3.27.1" -U tox-factor should install setuptools as well. The added dependency is only available inside the venv that tox creates. But it's required outside as well to build the sdist.

Thanks. Hopefully 2a682c9 is correct and sufficient.

musicinmybrain requested a review from a team as a code owner August 8, 2023 12:54

carolabadeer reviewed Aug 17, 2023

View reviewed changes

michael-k reviewed Oct 6, 2023

View reviewed changes

musicinmybrain and others added 4 commits October 6, 2023 07:53

Add setuptools as a test dependency for pkg_resources

2b0eef7

Required by the core tests in tests/test_local_sampling_benchmark.py.

Add tox environments for Python 3.12

50b2ede

Add Python 3.12 to GitHub unit testing workflow

e561e3b

Correct entry for Python 3.12 in UnitTesting.yaml

3989c7d

Co-authored-by: Carol Abadeer <60774943+carolabadeer@users.noreply.github.com>

musicinmybrain force-pushed the py312 branch from 4384ef4 to 3989c7d Compare October 6, 2023 11:53

musicinmybrain added 2 commits November 3, 2023 20:39

Merge branch 'master' into py312

0c32b79

Install setuptools alongside tox-factor

2a682c9

Ensure it is available for creating the sdist on Python 3.12+. aws#400 (comment)

srprash approved these changes Dec 1, 2023

View reviewed changes

srprash merged commit 2e07898 into aws:master Dec 1, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable testing on Python 3.12#400

Enable testing on Python 3.12#400
srprash merged 6 commits intoaws:masterfrom
musicinmybrain:py312

musicinmybrain commented Aug 8, 2023

Uh oh!

musicinmybrain commented Aug 8, 2023

Uh oh!

musicinmybrain commented Aug 9, 2023

Uh oh!

Uh oh!

carolabadeer Aug 17, 2023

Uh oh!

musicinmybrain Aug 18, 2023

Uh oh!

musicinmybrain Aug 18, 2023

Uh oh!

michael-k Oct 6, 2023

Uh oh!

musicinmybrain Oct 6, 2023

Uh oh!

michael-k Oct 6, 2023

Uh oh!

musicinmybrain commented Oct 6, 2023

Uh oh!

michael-k commented Oct 6, 2023

Uh oh!

musicinmybrain commented Nov 4, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

musicinmybrain commented Aug 8, 2023

Uh oh!

musicinmybrain commented Aug 8, 2023

Uh oh!

musicinmybrain commented Aug 9, 2023

Uh oh!

Uh oh!

carolabadeer Aug 17, 2023

Choose a reason for hiding this comment

Uh oh!

musicinmybrain Aug 18, 2023

Choose a reason for hiding this comment

Uh oh!

musicinmybrain Aug 18, 2023

Choose a reason for hiding this comment

Uh oh!

michael-k Oct 6, 2023

Choose a reason for hiding this comment

Uh oh!

musicinmybrain Oct 6, 2023

Choose a reason for hiding this comment

Uh oh!

michael-k Oct 6, 2023

Choose a reason for hiding this comment

Uh oh!

musicinmybrain commented Oct 6, 2023

Uh oh!

michael-k commented Oct 6, 2023

Uh oh!

musicinmybrain commented Nov 4, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants