Support tox 4.x for testing modularized distributions by mkoeppe · Pull Request #35208 · sagemath/sage

mkoeppe · 2023-02-27T03:08:14Z

📚 Description

Previous fixes for tox 4.x were not complete, as observed in #35095 (comment)

The files pkgs/*/tox.ini now requires tox >= 4.2.
tox 3.x and 4.x auto-provision a suitable version, so it is not immediately necessary to upgrade our tox package (or tighten the version requirements in build/pkgs/tox/spkg-configure.m4.

📝 Checklist

I have made sure that the title is self-explanatory and the description concisely explains the PR.
I have linked an issue or discussion.
I have created tests covering the changes.
I have updated the documentation accordingly.

⌛ Dependencies

…-sagewheels-nopypi-norequirements

codecov-commenter · 2023-02-27T09:24:15Z

Codecov Report

Patch coverage has no change and project coverage change: -0.01 ⚠️

Comparison is base (f449b14) 88.62% compared to head (eabf6fd) 88.61%.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop   #35208      +/-   ##
===========================================
- Coverage    88.62%   88.61%   -0.01%     
===========================================
  Files         2148     2148              
  Lines       398653   398653              
===========================================
- Hits        353308   353283      -25     
- Misses       45345    45370      +25

see 25 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

kwankyu · 2023-03-16T08:57:04Z

Running

make SAGE_WHEELS=yes sagemath_categories-build-deps && ./sage -sh -c '(cd pkgs/sagemath-categories && SAGE_NUM_THREADS=8 tox -v -v -v -e sagepython-sagewheels-nopypi-norequirements)'

I get

Writing sagemath-categories-10.0b4/setup.cfg
Creating tar archive
removing 'sagemath-categories-10.0b4' (and everything under it)
Backend: Wrote response {'return': 'sagemath-categories-10.0b4.tar.gz'} to /var/folders/td/fw1q9ljs311ggyph77rs53_40000gn/T/pep517_build_sdist-_eofd7b5.json
.pkg-sagepython-sagewheels-nopypi: 17005 I exit None (5.42 seconds) /Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-categories> python /usr/local/Cellar/tox/4.4.6/libexec/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta pid=2054 [tox/execute/api.py:275]
.pkg-sagepython-sagewheels-nopypi: 17006 D package .tmp/package/2/sagemath-categories-10.0b4.tar.gz links to .pkg-sagepython-sagewheels-nopypi/dist/sagemath-categories-10.0b4.tar.gz (/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-categories/.tox) [tox/util/file_view.py:36]
sagepython-sagewheels-nopypi-norequirements: 17007 W install_package_deps> python -I -m pip install 'sagemath-objects~=10.0b4' sagemath-repl [tox/tox_env/api.py:428]
Looking in links: file:///Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11.1/var/lib/sage/wheels
Processing /Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11.1/var/lib/sage/wheels/sagemath_objects-10.0b4-cp311-cp311-macosx_12_0_x86_64.whl
ERROR: Could not find a version that satisfies the requirement sagemath-repl (from versions: none)
ERROR: No matching distribution found for sagemath-repl
sagepython-sagewheels-nopypi-norequirements: 17779 C exit 1 (0.77 seconds) /Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-categories> python -I -m pip install 'sagemath-objects~=10.0b4' sagemath-repl pid=2056 [tox/execute/api.py:275]
.pkg-sagepython-sagewheels-nopypi: 17780 W _exit> python /usr/local/Cellar/tox/4.4.6/libexec/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta [tox/tox_env/api.py:428]
Backend: run command _exit with args {}
Backend: Wrote response {'return': 0} to /var/folders/td/fw1q9ljs311ggyph77rs53_40000gn/T/pep517__exit-ejch1cup.json
.pkg-sagepython-sagewheels-nopypi: 17783 I exit None (0.00 seconds) /Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-categories> python /usr/local/Cellar/tox/4.4.6/libexec/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta pid=2054 [tox/execute/api.py:275]
.pkg-sagepython-sagewheels-nopypi: 17872 D delete package /Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-categories/.tox/.tmp/package/2/sagemath-categories-10.0b4.tar.gz [tox/tox_env/python/virtual_env/package/pyproject.py:179]
  sagepython-sagewheels-nopypi-norequirements: FAIL code 1 (17.62 seconds)
  evaluation failed :( (17.71 seconds)

kwankyu · 2023-03-16T11:17:31Z

Running

 ./sage -sh -c '(cd pkgs/sagemath-standard && tox -v -v -v -e sagepython-sagewheels-nopypi)'

results in

...
sage -t --random-seed=233132637117582060430838751734210350117 /Users/kwankyu/GitHub/sage-dev/src/doc/de/a_tour_of_sage/index.rst  # 1 doctest failed
sage -t --random-seed=233132637117582060430838751734210350117 /Users/kwankyu/GitHub/sage-dev/src/sage/graphs/generators/families.py  # 3 doctests failed
sage -t --random-seed=233132637117582060430838751734210350117 /Users/kwankyu/GitHub/sage-dev/src/sage/graphs/graph_generators.py  # 1 doctest failed
sage -t --random-seed=233132637117582060430838751734210350117 /Users/kwankyu/GitHub/sage-dev/src/sage/data_structures/stream.py  # Timed out
sage -t --random-seed=233132637117582060430838751734210350117 /Users/kwankyu/GitHub/sage-dev/src/sage/misc/package.py  # 5 doctests failed
sage -t --random-seed=233132637117582060430838751734210350117 /Users/kwankyu/GitHub/sage-dev/src/sage/repl/ipython_extension.py  # 1 doctest failed
----------------------------------------------------------------------
Total time for all tests: 2173.7 seconds
    cpu time: 12668.0 seconds
    cumulative wall time: 16102.6 seconds
Features detected for doctesting: 4ti2,csdp,cvxopt,dvipng,gfan,graphviz,imagemagick,nauty,palp,pandoc,pdf2svg,pdftocairo,polytopes_db,sage.combinat,sage.geometry.polyhedron,sage.graphs,sage.groups,sage.misc.cython,sage.plot,sage.rings.number_field,sage.rings.padics,sage.rings.real_double,sage.symbolic,sagemath_doc_html,sphinx
pytest is not installed in the venv, skip checking tests that rely on it
sagepython-sagewheels-nopypi: 5220971 C exit 5 (2182.09 seconds) /Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard> sage -t -p --all pid=4907 [tox/execute/api.py:275]
.pkg-sagepython-sagewheels-nopypi: 5220972 W _exit> python /usr/local/Cellar/tox/4.4.6/libexec/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta [tox/tox_env/api.py:428]
Backend: run command _exit with args {}
Backend: Wrote response {'return': 0} to /var/folders/td/fw1q9ljs311ggyph77rs53_40000gn/T/pep517__exit-8m9b2g26.json
.pkg-sagepython-sagewheels-nopypi: 5220975 I exit None (0.00 seconds) /Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard> python /usr/local/Cellar/tox/4.4.6/libexec/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta pid=776 [tox/execute/api.py:275]
.pkg-sagepython-sagewheels-nopypi: 5221052 D delete package /Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/.tmp/package/1/sagemath-standard-10.0b4.tar.gz [tox/tox_env/python/virtual_env/package/pyproject.py:179]
  sagepython-sagewheels-nopypi: FAIL code 5 (5221.03=setup[3034.81]+cmd[2.00,2.12,2182.09] seconds)
  evaluation failed :( (5221.11 seconds)

mkoeppe · 2023-03-16T20:24:50Z

make SAGE_WHEELS=yes sagemath_categories-build-deps && ./sage -sh -c '(cd pkgs/sagemath-categories && SAGE_NUM_THREADS=8 tox -v -v -v -e sagepython-sagewheels-nopypi-norequirements)'

...

ERROR: Could not find a version that satisfies the requirement sagemath-repl (from versions: none)

Nice catch. Of course, sagemath-repl is not part of the build-deps of sagemath-categories unless SAGE_CHECK=yes is set. I'll update the instructions.

…ding sagemath_repl wheel

kwankyu · 2023-03-19T21:03:37Z

I ran

make SAGE_WHEELS=yes sagemath_categories-build-deps sagemath_repl && ./sage -sh -c '(cd pkgs/sagemath-categories && SAGE_NUM_THREADS=8 tox -v -v -v -e sagepython-sagewheels-nopypi-norequirements)'

after merging 10.0.beta5, and got

...
[sagemath_objects-10.0.beta5] * Creating venv isolated environment...
[sagemath_objects-10.0.beta5] * Installing packages in isolated environment... (cysignals >=1.10.2, cython >=0.29.21, <1.0, gmpy2 >=2.1.0, sage-setup ~= 10.0b5, sagemath-environment ~= 10.0b5, setuptools >=49.6.0, wheel >=0.36.2)
[sagemath_objects-10.0.beta5] Looking in links: file:///Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11.1/var/lib/sage/wheels
[sagemath_objects-10.0.beta5] ERROR: Could not find a version that satisfies the requirement sage-setup~=10.0b5 (from versions: 10.0b4)
[sagemath_objects-10.0.beta5] ERROR: No matching distribution found for sage-setup~=10.0b5
[sagemath_objects-10.0.beta5]

Then I did make wheels and tried again. Now it succeeded.

Was the initial failure expected?

mkoeppe · 2023-03-19T22:52:00Z

I ran

make SAGE_WHEELS=yes sagemath_categories-build-deps sagemath_repl && ./sage -sh -c '(cd pkgs/sagemath-categories && SAGE_NUM_THREADS=8 tox -v -v -v -e sagepython-sagewheels-nopypi-norequirements)'

after merging 10.0.beta5

These instructions from the top of tox.ini assume that no packages have been built yet. If sage_setup or other sagemath... packages were built already (in editable mode), this won't rebuild them.

make wheels (as shown in the developer's guide) takes care of first uninstalling the editable installations, which ensures that the wheels are available.

mkoeppe · 2023-03-20T01:36:29Z

sage -t --random-seed=233132637117582060430838751734210350117 /Users/kwankyu/GitHub/sage-dev/src/sage/misc/package.py # 5 doctests failed
sage -t --random-seed=233132637117582060430838751734210350117 /Users/kwankyu/GitHub/sage-dev/src/sage/repl/ipython_extension.py # 1 doctest failed

I think only these two may have something to do with this PR.

kwankyu · 2023-03-20T01:57:43Z

Now I am getting

ERROR: Could not find a version that satisfies the requirement scipy==1.10.1 (from versions: 1.9.3)
ERROR: No matching distribution found for scipy==1.10.1
sagepython-sagewheels-nopypi: 1496 C exit 1 (1.21 seconds) /Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard> python -I -m pip install -r requirements.txt pid=79397 [tox/execute/api.py:275]
  sagepython-sagewheels-nopypi: FAIL code 1 (1.27 seconds)
  evaluation failed :( (1.35 seconds)

from the run of

./sage -sh -c '(cd pkgs/sagemath-standard && tox -v -v -v -e sagepython-sagewheels-nopypi)'

mkoeppe · 2023-03-20T02:33:10Z

Is this after running make wheels?

kwankyu · 2023-03-20T02:40:13Z

Yes. It didn't make a difference.

Then I removed the old wheel scipy1.9.3, and make scipy. I am running the command again. It is now at

sagepython-sagewheels-nopypi: 98255 W install_package> python -I -m pip install --force-reinstall --no-deps /Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/.tmp/package/3/sagemath-standard-10.0b5.tar.gz [tox/tox_env/api.py:428]
Looking in links: file:///Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11.1/var/lib/sage/wheels
Processing ./.tox/.tmp/package/3/sagemath-standard-10.0b5.tar.gz
  Installing build dependencies ... \done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: sagemath-standard
  Building wheel for sagemath-standard (pyproject.toml) ... /

kwankyu · 2023-03-20T02:42:00Z

I see scipy-1.10.1-cp311-cp311-macosx_12_0_x86_64.whl in the wheels directory.

mkoeppe · 2023-03-20T03:02:54Z

OK, this happened because scipy is missing in build/pkgs/sagelib/dependencies. I think this was deliberate at one time, to allow the sagelib build to start before scipy has finished building.

mkoeppe · 2023-03-20T03:09:25Z

In fact, several other packages listed in src/setup.cfg.m4 under "SAGERUNTIME" and "DOC_DEPENDENCIES" should really be added

kwankyu · 2023-03-20T04:24:00Z

The run finished with

sage -t --random-seed=226979055037151283592527401300349772391 /Users/kwankyu/GitHub/sage-dev/src/doc/de/tutorial/latex.rst  # 1 doctest failed
sage -t --random-seed=226979055037151283592527401300349772391 /Users/kwankyu/GitHub/sage-dev/src/sage/graphs/generators/families.py  # 3 doctests failed
sage -t --random-seed=226979055037151283592527401300349772391 /Users/kwankyu/GitHub/sage-dev/src/sage/graphs/graph_generators.py  # 1 doctest failed
sage -t --random-seed=226979055037151283592527401300349772391 /Users/kwankyu/GitHub/sage-dev/src/sage/data_structures/stream.py  # Timed out
sage -t --random-seed=226979055037151283592527401300349772391 /Users/kwankyu/GitHub/sage-dev/src/sage/dynamics/arithmetic_dynamics/projective_ds.py  # Timed out
sage -t --random-seed=226979055037151283592527401300349772391 /Users/kwankyu/GitHub/sage-dev/src/sage/interfaces/gap_workspace.py  # 2 doctests failed
sage -t --random-seed=226979055037151283592527401300349772391 /Users/kwankyu/GitHub/sage-dev/src/sage/misc/package.py  # 5 doctests failed
sage -t --random-seed=226979055037151283592527401300349772391 /Users/kwankyu/GitHub/sage-dev/src/sage/repl/ipython_extension.py  # 1 doctest failed
----------------------------------------------------------------------
Total time for all tests: 2340.1 seconds
    cpu time: 13316.0 seconds
    cumulative wall time: 17139.9 seconds
Features detected for doctesting: 4ti2,csdp,cvxopt,dvipng,gfan,graphviz,imagemagick,nauty,palp,pandoc,pdf2svg,pdftocairo,polytopes_db,sage.combinat,sage.geometry.polyhedron,sage.graphs,sage.groups,sage.misc.cython,sage.plot,sage.rings.number_field,sage.rings.padics,sage.rings.real_double,sage.symbolic,sagemath_doc_html,sphinx
pytest is not installed in the venv, skip checking tests that rely on it
sagepython-sagewheels-nopypi: 5497901 C exit 5 (2351.72 seconds) /Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard> sage -t -p --all pid=39122 [tox/execute/api.py:275]
.pkg-sagepython-sagewheels-nopypi: 5497903 W _exit> python /usr/local/Cellar/tox/4.4.6/libexec/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta [tox/tox_env/api.py:428]
Backend: run command _exit with args {}
Backend: Wrote response {'return': 0} to /var/folders/td/fw1q9ljs311ggyph77rs53_40000gn/T/pep517__exit-6m1dxphn.json
.pkg-sagepython-sagewheels-nopypi: 5497907 I exit None (0.00 seconds) /Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard> python /usr/local/Cellar/tox/4.4.6/libexec/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta pid=35633 [tox/execute/api.py:275]
.pkg-sagepython-sagewheels-nopypi: 5498017 D delete package /Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/.tmp/package/3/sagemath-standard-10.0b5.tar.gz [tox/tox_env/python/virtual_env/package/pyproject.py:179]
  sagepython-sagewheels-nopypi: FAIL code 5 (5497.98=setup[3141.85]+cmd[2.07,2.34,2351.72] seconds)
  evaluation failed :( (5498.07 seconds)

kwankyu · 2023-03-20T04:31:05Z

sage -t --random-seed=233132637117582060430838751734210350117 /Users/kwankyu/GitHub/sage-dev/src/sage/misc/package.py # 5 doctests failed
sage -t --random-seed=233132637117582060430838751734210350117 /Users/kwankyu/GitHub/sage-dev/src/sage/repl/ipython_extension.py # 1 doctest failed

I think only these two may have something to do with this PR.

The tests pass in the normal sage install.

mkoeppe · 2023-03-20T04:33:09Z

Could you post the full log of the failures please?

kwankyu · 2023-03-20T04:41:15Z

Ah, I closed the terminal! I started running it again. I will take long to see the result.

kwankyu · 2023-03-20T07:05:42Z

sage -t --warn-long 80.0 --random-seed=311058290845287139560044406479667880892 /Users/kwankyu/GitHub/sage-dev/src/sage/repl/ipython_extension.py
**********************************************************************
File "/Users/kwankyu/GitHub/sage-dev/src/sage/repl/ipython_extension.py", line 353, in sage.repl.ipython_extension.SageMagics.cython
Failed example:
    shell.run_cell('''
    %%cython
    def f():
        print('test')
    ''')
Expected nothing
Got:
    doctest:warning
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/bin/sage-runtests", line 154, in <module>
        err = DC.run()
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/control.py", line 1384, in run
        self.run_doctests()
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/control.py", line 1059, in run_doctests
        self.dispatcher.dispatch()
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/forker.py", line 2017, in dispatch
        self.parallel_dispatch()
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/forker.py", line 1912, in parallel_dispatch
        w.start()  # This might take some time
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/forker.py", line 2186, in start
        super().start()
      File "/Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11.1/lib/python3.11/multiprocessing/process.py", line 121, in start
        self._popen = self._Popen(self)
      File "/Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11.1/lib/python3.11/multiprocessing/context.py", line 224, in _Popen
        return _default_context.get_context().Process._Popen(process_obj)
      File "/Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11.1/lib/python3.11/multiprocessing/context.py", line 281, in _Popen
        return Popen(process_obj)
      File "/Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11.1/lib/python3.11/multiprocessing/popen_fork.py", line 19, in __init__
        self._launch(process_obj)
      File "/Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11.1/lib/python3.11/multiprocessing/popen_fork.py", line 71, in _launch
        code = process_obj._bootstrap(parent_sentinel=child_r)
      File "/Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11.1/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
        self.run()
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/forker.py", line 2158, in run
        task(self.options, self.outtmpfile, msgpipe, self.result_queue)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/forker.py", line 2488, in __call__
        doctests, extras = self._run(runner, options, results)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/forker.py", line 2540, in _run
        result = runner.run(test)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/forker.py", line 866, in run
        return self._run(test, compileflags, out)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/forker.py", line 695, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/forker.py", line 1093, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.repl.ipython_extension.SageMagics.cython[2]>", line 1, in <module>
        shell.run_cell('''
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/repl/interpreter.py", line 390, in run_cell
        super().run_cell(*args, **kwds)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 2940, in run_cell
        result = self._run_cell(
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 2995, in _run_cell
        return runner(coro)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/IPython/core/async_helpers.py", line 129, in _pseudo_sync_runner
        coro.send(None)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 3194, in run_cell_async
        has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 3373, in run_ast_nodes
        if await self.run_code(code, result, async_=asy):
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 3433, in run_code
        exec(code_obj, self.user_global_ns, self.user_ns)
      File "<ipython-input-1-b9ac10b76270>", line 1, in <module>
        get_ipython().run_cell_magic('cython', '', "def f():\n    print('test')\n")
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 2417, in run_cell_magic
        result = fn(*args, **kwargs)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/repl/ipython_extension.py", line 362, in cython
        return cython_compile(cell)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/sage/misc/cython.py", line 673, in cython_compile
        return cython_import_all(tmpfile, get_globals(), **kwds)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/sage/misc/cython.py", line 563, in cython_import_all
        m = cython_import(filename, **kwds)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/sage/misc/cython.py", line 534, in cython_import
        name, build_dir = cython(filename, **kwds)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/sage/misc/cython.py", line 291, in cython
        standard_libs, standard_libdirs, standard_includes, aliases = _standard_libs_libdirs_incdirs_aliases()
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/sage/misc/cython.py", line 60, in _standard_libs_libdirs_incdirs_aliases
        standard_incdirs = sage_include_directories() + aliases["CBLAS_INCDIR"] + aliases["NTL_INCDIR"]
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/env.py", line 307, in sage_include_directories
        import distutils.sysconfig
      File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
      File "<frozen importlib._bootstrap>", line 1128, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
      File "<frozen importlib._bootstrap>", line 1140, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 1080, in _find_spec
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/_distutils_hack/__init__.py", line 97, in find_spec
        return method()
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/_distutils_hack/__init__.py", line 108, in spec_for_distutils
        mod = importlib.import_module('setuptools._distutils')
      File "/Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11.1/lib/python3.11/importlib/__init__.py", line 126, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
      File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
      File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
      File "<frozen importlib._bootstrap>", line 1128, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
      File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
      File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 940, in exec_module
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/setuptools/__init__.py", line 16, in <module>
        import setuptools.version
      File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
      File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 940, in exec_module
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/setuptools/version.py", line 1, in <module>
        import pkg_resources
      File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
      File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 940, in exec_module
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/pkg_resources/__init__.py", line 3248, in <module>
        @_call_aside
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/pkg_resources/__init__.py", line 3223, in _call_aside
        f(*args, **kwargs)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/pkg_resources/__init__.py", line 3274, in _initialize_master_working_set
        tuple(dist.activate(replace=False) for dist in working_set)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/pkg_resources/__init__.py", line 3274, in <genexpr>
        tuple(dist.activate(replace=False) for dist in working_set)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/pkg_resources/__init__.py", line 2790, in activate
        self.insert_on(path, replace=replace)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/pkg_resources/__init__.py", line 2922, in insert_on
        self.check_version_conflict()
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/pkg_resources/__init__.py", line 2963, in check_version_conflict
        issue_warning(
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/pkg_resources/__init__.py", line 3076, in issue_warning
        warnings.warn(stacklevel=level + 1, *args, **kw)
      File "/Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11.1/lib/python3.11/warnings.py", line 109, in _showwarnmsg
        sw(msg.message, msg.category, msg.filename, msg.lineno,
    :
    UserWarning: Module sage was already imported from None, but /Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard is being added to sys.path
    ld: warning: -undefined dynamic_lookup may not work with chained fixups
**********************************************************************
1 item had failures:
   1 of   5 in sage.repl.ipython_extension.SageMagics.cython
    [93 tests, 1 failure, 14.79 s]

kwankyu · 2023-03-20T07:07:36Z

sage -t --warn-long 80.0 --random-seed=311058290845287139560044406479667880892 /Users/kwankyu/GitHub/sage-dev/src/sage/misc/package.py
**********************************************************************
File "/Users/kwankyu/GitHub/sage-dev/src/sage/misc/package.py", line 389, in sage.misc.package.installed_packages
Failed example:
    sorted(installed_packages().keys())  # optional - sage_spkg
Expected:
    [...'gmpy2', ...'sage_conf', ...]
Got:
    ['arb',
     'brial',
     'bzip2',
     'cddlib',
     'cliquer',
     'combinatorial_designs',
     'conway_polynomials',
     'eclib',
     'ecm',
     'elliptic_curves',
     'fflas_ffpack',
     'flint',
     'flintqs',
     'gap',
     'gf2x',
     'gfortran',
     'giac',
     'givaro',
     'glpk',
     'gnulib',
     'graphs',
     'gsl',
     'iml',
     'jmol',
     'lcalc',
     'libbraiding',
     'libhomfly',
     'linbox',
     'lrcalc',
     'm4ri',
     'm4rie',
     'mathjax',
     'maxima',
     'mpc',
     'openblas',
     'palp',
     'pari',
     'pari_galdata',
     'pari_seadata_small',
     'patchelf',
     'pcre',
     'planarity',
     'polytopes_db',
     'ppl',
     'pplpy_doc',
     'rw',
     'sagemath_doc_html',
     'singular',
     'sqlite',
     'suitesparse',
     'symmetrica',
     'sympow',
     'tachyon',
     'threejs']
**********************************************************************
File "/Users/kwankyu/GitHub/sage-dev/src/sage/misc/package.py", line 391, in sage.misc.package.installed_packages
Failed example:
    installed_packages()['gmpy2']  # optional - sage_spkg, random
Exception raised:
    Traceback (most recent call last):
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/forker.py", line 695, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/forker.py", line 1093, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.misc.package.installed_packages[1]>", line 1, in <module>
        installed_packages()['gmpy2']  # optional - sage_spkg, random
        ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
    KeyError: 'gmpy2'
**********************************************************************
File "/Users/kwankyu/GitHub/sage-dev/src/sage/misc/package.py", line 624, in sage.misc.package.package_manifest
Failed example:
    sagetex_manifest = package_manifest('sagetex')  # optional - sage_spkg
Exception raised:
    Traceback (most recent call last):
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/forker.py", line 695, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/forker.py", line 1093, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.misc.package.package_manifest[1]>", line 1, in <module>
        sagetex_manifest = package_manifest('sagetex')  # optional - sage_spkg
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/misc/package.py", line 637, in package_manifest
        version = installed_packages()[package]
                  ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
    KeyError: 'sagetex'
**********************************************************************
File "/Users/kwankyu/GitHub/sage-dev/src/sage/misc/package.py", line 625, in sage.misc.package.package_manifest
Failed example:
    sagetex_manifest['package_name'] == 'sagetex'  # optional - sage_spkg
Exception raised:
    Traceback (most recent call last):
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/forker.py", line 695, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/forker.py", line 1093, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.misc.package.package_manifest[2]>", line 1, in <module>
        sagetex_manifest['package_name'] == 'sagetex'  # optional - sage_spkg
        ^^^^^^^^^^^^^^^^
    NameError: name 'sagetex_manifest' is not defined
**********************************************************************
File "/Users/kwankyu/GitHub/sage-dev/src/sage/misc/package.py", line 627, in sage.misc.package.package_manifest
Failed example:
    'files' in sagetex_manifest  # optional - sage_spkg
Exception raised:
    Traceback (most recent call last):
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/forker.py", line 695, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/forker.py", line 1093, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.misc.package.package_manifest[3]>", line 1, in <module>
        'files' in sagetex_manifest  # optional - sage_spkg
                   ^^^^^^^^^^^^^^^^
    NameError: name 'sagetex_manifest' is not defined
**********************************************************************
2 items had failures:
   2 of   3 in sage.misc.package.installed_packages
   3 of   6 in sage.misc.package.package_manifest
    [56 tests, 5 failures, 11.24 s]

kwankyu · 2023-04-01T12:56:15Z

With the latest beta, I get from ./sage -sh -c '(cd pkgs/sagemath-standard && tox -v -v -v -e sagepython-sagewheels-nopypi)'

sage -t --random-seed=160851146271410412654781755279246610499 /Users/kwankyu/GitHub/sage-dev/src/sage/tests/cmdline.py
    [217 tests, 76.30 s]
----------------------------------------------------------------------
sage -t --random-seed=160851146271410412654781755279246610499 /Users/kwankyu/GitHub/sage-dev/src/sage/graphs/generators/families.py  # 3 doctests failed
sage -t --random-seed=160851146271410412654781755279246610499 /Users/kwankyu/GitHub/sage-dev/src/sage/graphs/graph_generators.py  # 1 doctest failed
sage -t --random-seed=160851146271410412654781755279246610499 /Users/kwankyu/GitHub/sage-dev/src/sage/misc/package.py  # 5 doctests failed
sage -t --random-seed=160851146271410412654781755279246610499 /Users/kwankyu/GitHub/sage-dev/src/sage/repl/ipython_extension.py  # 1 doctest failed
----------------------------------------------------------------------
Total time for all tests: 1788.2 seconds
    cpu time: 10438.1 seconds
    cumulative wall time: 13431.9 seconds
Features detected for doctesting: 4ti2,csdp,cvxopt,dvipng,gfan,graphviz,imagemagick,nauty,palp,pandoc,pdf2svg,pdftocairo,polytopes_db,sage.combinat,sage.geometry.polyhedron,sage.graphs,sage.groups,sage.libs.pari,sage.misc.cython,sage.plot,sage.rings.number_field,sage.rings.padics,sage.rings.real_double,sage.rings.real_mpfr,sage.symbolic,sagemath_doc_html,sphinx
pytest is not installed in the venv, skip checking tests that rely on it
sagepython-sagewheels-nopypi: 4880330 C exit 1 (1799.78 seconds) /Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard> sage -t -p --all pid=15086 [tox/execute/api.py:275]
.pkg-sagepython-sagewheels-nopypi: 4880331 W _exit> python /usr/local/Cellar/tox/4.4.6/libexec/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta [tox/tox_env/api.py:428]
Backend: run command _exit with args {}
Backend: Wrote response {'return': 0} to /var/folders/td/fw1q9ljs311ggyph77rs53_40000gn/T/pep517__exit-408penwy.json
.pkg-sagepython-sagewheels-nopypi: 4880334 I exit None (0.00 seconds) /Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard> python /usr/local/Cellar/tox/4.4.6/libexec/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta pid=873 [tox/execute/api.py:275]
.pkg-sagepython-sagewheels-nopypi: 4880422 D delete package /Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/.tmp/package/5/sagemath-standard-10.0b6.tar.gz [tox/tox_env/python/virtual_env/package/pyproject.py:179]
  sagepython-sagewheels-nopypi: FAIL code 1 (4880.31=setup[3076.63]+cmd[1.81,2.08,1799.78] seconds)
  evaluation failed :( (4880.42 seconds)

What does evaluation failed mean?

SageMath version 10.0.rc1, Release Date: 2023-04-28

…in a sub-python

mkoeppe · 2023-05-01T07:34:48Z

What does evaluation failed mean?

This message comes from tox, in response to the error exit of the doctester.

mkoeppe · 2023-05-01T07:38:09Z

sage -t --random-seed=160851146271410412654781755279246610499 /Users/kwankyu/GitHub/sage-dev/src/sage/graphs/generators/families.py # 3 doctests failed
sage -t --random-seed=160851146271410412654781755279246610499 /Users/kwankyu/GitHub/sage-dev/src/sage/graphs/graph_generators.py # 1 doctest failed

These two were fixed in #35250, I think.

sage -t --random-seed=160851146271410412654781755279246610499 /Users/kwankyu/GitHub/sage-dev/src/sage/misc/package.py # 5 doctests failed
sage -t --random-seed=160851146271410412654781755279246610499 /Users/kwankyu/GitHub/sage-dev/src/sage/repl/ipython_extension.py # 1 doctest failed

I've fixed these two

kwankyu · 2023-05-02T08:24:59Z

pkgs/sagemath-categories/tox.ini

 #
-# pkgs/sagemath-categories/.tox/sagepython/bin/python
+#   pkgs/sagemath-categories/.tox/sagepython-sagewheels-nopypi-norequirements/bin/sage
 #


Remove initial two spaces.

kwankyu · 2023-05-02T08:25:28Z

src/sage/misc/package.py

+    the installation record for this package. (We do not test for installation
+    records of Python packages. Our ``SAGE_VENV`` is not necessarily the
+    main Sage venv; it could be a user-created venv or a venv created by tox.)



Add :: at the end.

kwankyu · 2023-05-02T08:35:45Z

Otherwise, all works well.

mkoeppe · 2023-05-02T20:10:32Z

Thanks! I've made these changes.

github-actions · 2023-05-02T21:26:31Z

Documentation preview for this PR is ready! 🎉
Built with commit: 0c8bb55

kwankyu

Thanks. All works well for me.

mkoeppe · 2023-05-03T04:39:34Z

Thank you!

Matthias Koeppe added 8 commits February 26, 2023 18:52

pkgs/*/tox.ini, src/doc/en/developer: Update instructions for testing

dd0575d

pkgs/sagemath-categories/tox.ini: Change default toxenv to sagepython…

829d567

…-sagewheels-nopypi-norequirements

pkgs/sagemath-polyhedra/tox.ini: Update for tox 4

379c6a4

pkgs/sagemath-*/tox.ini: Update for tox 4

513eac0

pkgs/sagemath-categories/tox.ini: Fix up

4a35130

pkgs/sagemath-*/tox.ini: Configure testenv:.tox

a156e2a

pkgs/sagemath-*/tox.ini: Configure testenv:.tox (fixup)

353e625

pkgs/sagemath-categories/tox.ini: fix up

b48a24c

Matthias Koeppe added 2 commits March 17, 2023 19:12

pkgs/sagemath-categories/tox.ini: Update instructions to include buil…

9597136

…ding sagemath_repl wheel

Merge remote-tracking branch 'upstream/develop' into pkgs_tox_4

eabf6fd

Matthias Koeppe added 3 commits April 30, 2023 19:33

Merge tag '10.0.rc1' into pkgs_tox_4

d205f56

SageMath version 10.0.rc1, Release Date: 2023-04-28

src/sage/misc/package.py: Make doctests work within venvs

cf8f92e

pkgs/sagemath-standard/tox.ini: change_dir to fix shadowing by sage/ …

c350f06

…in a sub-python

src/sage/misc/package.py: Make doctests work within venvs, part 2

b46a2d3

kwankyu reviewed May 2, 2023

View reviewed changes

Fixups

0c8bb55

kwankyu approved these changes May 3, 2023

View reviewed changes

kwankyu added s: positive review and removed s: needs review labels May 3, 2023

vbraun merged commit 9de8e26 into sagemath:develop May 22, 2023

Uh oh!

Conversation

mkoeppe commented Feb 27, 2023

📚 Description

📝 Checklist

⌛ Dependencies

Uh oh!

codecov-commenter commented Feb 27, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

kwankyu commented Mar 16, 2023

Uh oh!

kwankyu commented Mar 16, 2023

Uh oh!

mkoeppe commented Mar 16, 2023

Uh oh!

kwankyu commented Mar 19, 2023

Uh oh!

mkoeppe commented Mar 19, 2023

Uh oh!

mkoeppe commented Mar 20, 2023

Uh oh!

kwankyu commented Mar 20, 2023

Uh oh!

mkoeppe commented Mar 20, 2023

Uh oh!

kwankyu commented Mar 20, 2023

Uh oh!

kwankyu commented Mar 20, 2023

Uh oh!

mkoeppe commented Mar 20, 2023

Uh oh!

mkoeppe commented Mar 20, 2023

Uh oh!

kwankyu commented Mar 20, 2023

Uh oh!

kwankyu commented Mar 20, 2023

Uh oh!

mkoeppe commented Mar 20, 2023

Uh oh!

kwankyu commented Mar 20, 2023

Uh oh!

kwankyu commented Mar 20, 2023

Uh oh!

kwankyu commented Mar 20, 2023

Uh oh!

kwankyu commented Apr 1, 2023

Uh oh!

mkoeppe commented May 1, 2023

Uh oh!

mkoeppe commented May 1, 2023

Uh oh!

kwankyu May 2, 2023

Choose a reason for hiding this comment

Uh oh!

kwankyu May 2, 2023

Choose a reason for hiding this comment

Uh oh!

kwankyu commented May 2, 2023

Uh oh!

mkoeppe commented May 2, 2023

Uh oh!

github-actions bot commented May 2, 2023

Uh oh!

kwankyu left a comment

Choose a reason for hiding this comment

Uh oh!

mkoeppe commented May 3, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codecov-commenter commented Feb 27, 2023 •

edited

Loading