Skip to content

imgmath: Failed to build an image for equations during dvisvgm #9868

@kloczek

Description

@kloczek

Describe the bug

Pytest is failing in tests/test_util_inspect.py with error message about missing _testcapi module.
Second fail is in tests/test_ext_math.py::test_imgmath_svg unit.

How to Reproduce

I'm builing all my python modules as rpm packages so I'm unig below methodology:

  • "setup.py build"
  • "setup.py install --root </install/prefix>"
  • "pytest with PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>

Expected behavior

No errors or fails.

Your project

N/A

Screenshots

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-sphinx-4.3.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-sphinx-4.3.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.12, pytest-6.2.5, py-1.10.0, pluggy-0.13.1
libraries: Sphinx-4.3.0, docutils-0.17.1
base tempdir: /tmp/pytest-of-tkloczko/pytest-13
rootdir: /home/tkloczko/rpmbuild/BUILD/Sphinx-4.3.0, configfile: setup.cfg, testpaths: tests
plugins: rerunfailures-9.1.1, cov-2.12.1, forked-1.3.0, xdist-2.3.0, flake8-1.0.7, shutil-1.7.0, virtualenv-1.7.0, trio-0.7.0, mock-3.6.1, timeout-2.0.1, anyio-3.3.1, tornado-0.8.1, asyncio-0.15.1, httpbin-1.0.0, hypothesis-6.14.6, subtests-0.5.0, localserver-0.5.0, flaky-3.7.0
collected 1707 items / 1 error / 1706 selected

================================================================================== ERRORS ==================================================================================
_______________________________________________________________ ERROR collecting tests/test_util_inspect.py ________________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/Sphinx-4.3.0/tests/test_util_inspect.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib64/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_util_inspect.py:19: in <module>
    import _testcapi
E   ModuleNotFoundError: No module named '_testcapi'
========================================================================= short test summary info ==========================================================================
ERROR tests/test_util_inspect.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
============================================================================ 1 error in 19.48s =============================================================================

After ignore tests/test_util_inspect.py:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-sphinx-4.3.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-sphinx-4.3.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra --ignore tests/test_util_inspect.py
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.12, pytest-6.2.5, py-1.10.0, pluggy-0.13.1
libraries: Sphinx-4.3.0, docutils-0.17.1
base tempdir: /tmp/pytest-of-tkloczko/pytest-16
rootdir: /home/tkloczko/rpmbuild/BUILD/Sphinx-4.3.0, configfile: setup.cfg, testpaths: tests
plugins: rerunfailures-9.1.1, cov-2.12.1, forked-1.3.0, xdist-2.3.0, flake8-1.0.7, shutil-1.7.0, virtualenv-1.7.0, trio-0.7.0, mock-3.6.1, timeout-2.0.1, anyio-3.3.1, tornado-0.8.1, asyncio-0.15.1, httpbin-1.0.0, hypothesis-6.14.6, subtests-0.5.0, localserver-0.5.0, flaky-3.7.0
collected 1707 items

tests/test_api_translator.py ...........                                                                                                                             [  0%]
tests/test_application.py .......                                                                                                                                    [  1%]
tests/test_build.py ..........                                                                                                                                       [  1%]
tests/test_build_changes.py ..                                                                                                                                       [  1%]
tests/test_build_dirhtml.py .                                                                                                                                        [  1%]
tests/test_build_epub.py ..........s                                                                                                                                 [  2%]
tests/test_build_gettext.py .......                                                                                                                                  [  2%]
tests/test_build_html.py ........................................................................................................................................... [ 11%]
.................................................................................................................................................................... [ 20%]
.................................................................................................................................................................... [ 30%]
............................................................                                                                                                         [ 33%]
tests/test_build_latex.py .................................................................................                                                          [ 38%]
tests/test_build_linkcheck.py .............................                                                                                                          [ 40%]
tests/test_build_manpage.py ......                                                                                                                                   [ 40%]
tests/test_build_texinfo.py ......                                                                                                                                   [ 40%]
tests/test_build_text.py ...............                                                                                                                             [ 41%]
tests/test_builder.py ..                                                                                                                                             [ 41%]
tests/test_catalogs.py ...                                                                                                                                           [ 42%]
tests/test_config.py ..................................                                                                                                              [ 44%]
tests/test_correct_year.py ...                                                                                                                                       [ 44%]
tests/test_directive_code.py .......................................                                                                                                 [ 46%]
tests/test_directive_only.py .                                                                                                                                       [ 46%]
tests/test_directive_other.py .......                                                                                                                                [ 46%]
tests/test_directive_patch.py ...                                                                                                                                    [ 47%]
tests/test_docutilsconf.py ..                                                                                                                                        [ 47%]
tests/test_domain_c.py ..........................                                                                                                                    [ 48%]
tests/test_domain_cpp.py .................................                                                                                                           [ 50%]
tests/test_domain_js.py .........                                                                                                                                    [ 51%]
tests/test_domain_py.py ...............................................                                                                                              [ 54%]
tests/test_domain_rst.py ........                                                                                                                                    [ 54%]
tests/test_domain_std.py ...................                                                                                                                         [ 55%]
tests/test_environment.py ....                                                                                                                                       [ 55%]
tests/test_environment_indexentries.py ........                                                                                                                      [ 56%]
tests/test_environment_toctree.py .........                                                                                                                          [ 56%]
tests/test_errors.py ..                                                                                                                                              [ 56%]
tests/test_events.py ..                                                                                                                                              [ 57%]
tests/test_ext_apidoc.py ........................                                                                                                                    [ 58%]
tests/test_ext_autodoc.py ...................................................s..................                                                                     [ 62%]
tests/test_ext_autodoc_autoattribute.py ..............                                                                                                               [ 63%]
tests/test_ext_autodoc_autoclass.py .................                                                                                                                [ 64%]
tests/test_ext_autodoc_autodata.py ........                                                                                                                          [ 64%]
tests/test_ext_autodoc_autofunction.py .............                                                                                                                 [ 65%]
tests/test_ext_autodoc_automodule.py .....                                                                                                                           [ 65%]
tests/test_ext_autodoc_autoproperty.py ...                                                                                                                           [ 66%]
tests/test_ext_autodoc_configs.py .........................                                                                                                          [ 67%]
tests/test_ext_autodoc_events.py ......                                                                                                                              [ 67%]
tests/test_ext_autodoc_mock.py .......                                                                                                                               [ 68%]
tests/test_ext_autodoc_preserve_defaults.py .                                                                                                                        [ 68%]
tests/test_ext_autodoc_private_members.py ...                                                                                                                        [ 68%]
tests/test_ext_autosectionlabel.py ...                                                                                                                               [ 68%]
tests/test_ext_autosummary.py .......................                                                                                                                [ 70%]
tests/test_ext_coverage.py ....                                                                                                                                      [ 70%]
tests/test_ext_doctest.py ......                                                                                                                                     [ 70%]
tests/test_ext_duration.py .                                                                                                                                         [ 70%]
tests/test_ext_githubpages.py ...                                                                                                                                    [ 70%]
tests/test_ext_graphviz.py .....                                                                                                                                     [ 71%]
tests/test_ext_ifconfig.py .                                                                                                                                         [ 71%]
tests/test_ext_inheritance_diagram.py ......                                                                                                                         [ 71%]
tests/test_ext_intersphinx.py ...................                                                                                                                    [ 72%]
tests/test_ext_math.py .F..............                                                                                                                              [ 73%]
tests/test_ext_napoleon.py .................                                                                                                                         [ 74%]
tests/test_ext_napoleon_docstring.py ................................................                                                                                [ 77%]
tests/test_ext_napoleon_iterators.py ................                                                                                                                [ 78%]
tests/test_ext_todo.py ...                                                                                                                                           [ 78%]
tests/test_ext_viewcode.py .....                                                                                                                                     [ 78%]
tests/test_extension.py .                                                                                                                                            [ 78%]
tests/test_highlighting.py .....                                                                                                                                     [ 79%]
tests/test_intl.py ...................................................                                                                                               [ 82%]
tests/test_locale.py ...                                                                                                                                             [ 82%]
tests/test_markup.py ....................................                                                                                                            [ 84%]
tests/test_metadata.py .                                                                                                                                             [ 84%]
tests/test_parser.py .                                                                                                                                               [ 84%]
tests/test_project.py ...                                                                                                                                            [ 84%]
tests/test_pycode.py ........                                                                                                                                        [ 85%]
tests/test_pycode_ast.py ..........................................                                                                                                  [ 87%]
tests/test_pycode_parser.py .............................                                                                                                            [ 89%]
tests/test_quickstart.py ..........                                                                                                                                  [ 89%]
tests/test_roles.py .                                                                                                                                                [ 90%]
tests/test_search.py ...........                                                                                                                                     [ 90%]
tests/test_setup_command.py .......                                                                                                                                  [ 91%]
tests/test_smartquotes.py ..........                                                                                                                                 [ 91%]
tests/test_templating.py ...                                                                                                                                         [ 91%]
tests/test_theming.py .......                                                                                                                                        [ 92%]
tests/test_toctree.py ...                                                                                                                                            [ 92%]
tests/test_transforms_post_transforms.py ...                                                                                                                         [ 92%]
tests/test_transforms_post_transforms_code.py ...                                                                                                                    [ 92%]
tests/test_util.py ........                                                                                                                                          [ 93%]
tests/test_util_docstrings.py ...                                                                                                                                    [ 93%]
tests/test_util_docutils.py ...                                                                                                                                      [ 93%]
tests/test_util_fileutil.py ..                                                                                                                                       [ 93%]
tests/test_util_i18n.py .......                                                                                                                                      [ 94%]
tests/test_util_images.py ....                                                                                                                                       [ 94%]
tests/test_util_inventory.py ...                                                                                                                                     [ 94%]
tests/test_util_jsdump.py .                                                                                                                                          [ 94%]
tests/test_util_logging.py .................                                                                                                                         [ 95%]
tests/test_util_matching.py ..                                                                                                                                       [ 95%]
tests/test_util_nodes.py ..........................                                                                                                                  [ 97%]
tests/test_util_rst.py .......                                                                                                                                       [ 97%]
tests/test_util_template.py ..                                                                                                                                       [ 97%]
tests/test_util_typing.py .........ss....ss.......s..                                                                                                                [ 99%]
tests/test_versioning.py ..........                                                                                                                                  [ 99%]
tests/test_writer_latex.py .                                                                                                                                         [100%]

================================================================================= FAILURES =================================================================================
_____________________________________________________________________________ test_imgmath_svg _____________________________________________________________________________

command = ['dvisvgm', '-o', '/tmp/tmp9jki3pqi/math.svg', '--no-fonts', '/tmp/tmp9jki3pqi/math.dvi'], name = 'dvisvgm'

    def convert_dvi_to_image(command: List[str], name: str) -> Tuple[str, str]:
        """Convert DVI file to specific image format."""
        try:
>           ret = subprocess.run(command, stdout=PIPE, stderr=PIPE, check=True, encoding='ascii')

sphinx/ext/imgmath.py:152:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

input = None, capture_output = False, timeout = None, check = True, popenargs = (['dvisvgm', '-o', '/tmp/tmp9jki3pqi/math.svg', '--no-fonts', '/tmp/tmp9jki3pqi/math.dvi'],)
kwargs = {'encoding': 'ascii', 'stderr': -1, 'stdout': -1}, process = <subprocess.Popen object at 0x7f94bc49da30>
stdout = 'While reading gs_ll3.ps:\nError: /undefinedfilename in --resourceforall--\nOperand stack:\n   (gs_icc.ps\\000gs_mex_e...o)(G)--\nCurrent allocation mode is global\nLast OS error: No such file or directory\nCurrent file position is 18949\n'
stderr = 'PostScript error: invalidaccess in def\nOperand stack:\n    gs_icc.ps\n', retcode = 254

    def run(*popenargs,
            input=None, capture_output=False, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.

        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them.

        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.

        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.

        There is an optional argument "input", allowing you to
        pass bytes or a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.

        By default, all communication is in bytes, and therefore any "input" should
        be bytes, and the stdout and stderr will be bytes. If in text mode, any
        "input" should be a string, and stdout and stderr will be strings decoded
        according to locale encoding, or by "encoding" if set. Text mode is
        triggered by setting any of text, encoding, errors or universal_newlines.

        The other arguments are the same as for the Popen constructor.
        """
        if input is not None:
            if kwargs.get('stdin') is not None:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE

        if capture_output:
            if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
                raise ValueError('stdout and stderr arguments may not be used '
                                 'with capture_output.')
            kwargs['stdout'] = PIPE
            kwargs['stderr'] = PIPE

        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired as exc:
                process.kill()
                if _mswindows:
                    # Windows accumulates the output in a single blocking
                    # read() call run on child threads, with the timeout
                    # being done in a join() on those threads.  communicate()
                    # _after_ kill() is required to collect that and add it
                    # to the exception.
                    exc.stdout, exc.stderr = process.communicate()
                else:
                    # POSIX _communicate already populated the output so
                    # far into the TimeoutExpired exception.
                    process.wait()
                raise
            except:  # Including KeyboardInterrupt, communicate handled that.
                process.kill()
                # We don't call process.wait() as .__exit__ does that for us.
                raise
            retcode = process.poll()
            if check and retcode:
>               raise CalledProcessError(retcode, process.args,
                                         output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command '['dvisvgm', '-o', '/tmp/tmp9jki3pqi/math.svg', '--no-fonts', '/tmp/tmp9jki3pqi/math.dvi']' returned non-zero exit status 254.

/usr/lib64/python3.8/subprocess.py:516: CalledProcessError

During handling of the above exception, another exception occurred:

app = <SphinxTestApp buildername='html'>, status = <_io.StringIO object at 0x7f94bc4b1b80>, warning = <_io.StringIO object at 0x7f94bc4b1af0>

    @pytest.mark.skipif(not has_binary('dvisvgm'),
                        reason='Requires dvisvgm" binary')
    @pytest.mark.sphinx('html', testroot='ext-math-simple',
                        confoverrides={'extensions': ['sphinx.ext.imgmath'],
                                       'imgmath_image_format': 'svg'})
    def test_imgmath_svg(app, status, warning):
>       app.builder.build_all()

tests/test_ext_math.py:55:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
sphinx/builders/__init__.py:258: in build_all
    self.build(None, summary=__('all source files'), method='all')
sphinx/builders/__init__.py:358: in build
    self.write(docnames, list(updated_docnames), method)
sphinx/builders/__init__.py:532: in write
    self._write_serial(sorted(docnames))
sphinx/builders/__init__.py:542: in _write_serial
    self.write_doc(docname, doctree)
sphinx/builders/html/__init__.py:626: in write_doc
    self.docwriter.write(doctree, destination)
/usr/lib/python3.8/site-packages/docutils/writers/__init__.py:78: in write
    self.translate()
sphinx/writers/html.py:71: in translate
    self.document.walkabout(visitor)
/usr/lib/python3.8/site-packages/docutils/nodes.py:227: in walkabout
    if child.walkabout(visitor):
/usr/lib/python3.8/site-packages/docutils/nodes.py:227: in walkabout
    if child.walkabout(visitor):
/usr/lib/python3.8/site-packages/docutils/nodes.py:219: in walkabout
    visitor.dispatch_visit(self)
sphinx/util/docutils.py:477: in dispatch_visit
    method(node)
sphinx/writers/html5.py:802: in visit_math_block
    visit(self, node)
sphinx/ext/imgmath.py:320: in html_visit_displaymath
    fname, depth = render_math(self, latex)
sphinx/ext/imgmath.py:263: in render_math
    imgpath, depth = convert_dvi_to_svg(dvipath, self.builder)
sphinx/ext/imgmath.py:199: in convert_dvi_to_svg
    stdout, stderr = convert_dvi_to_image(command, name)
sphinx/ext/imgmath.py:160: in convert_dvi_to_image
    raise MathExtError('%s exited with error' % name, exc.stderr, exc.stdout) from exc
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = MathExtError('dvisvgm exited with error', 'PostScript error: invalidaccess in def\nOperand stack:\n    gs_icc.ps\n', '...)(G)--\nCurrent allocation mode is global\nLast OS error: No such file or directory\nCurrent file position is 18949\n')
msg = 'dvisvgm exited with error', stderr = 'PostScript error: invalidaccess in def\nOperand stack:\n    gs_icc.ps\n'
stdout = 'While reading gs_ll3.ps:\nError: /undefinedfilename in --resourceforall--\nOperand stack:\n   (gs_icc.ps\\000gs_mex_e...o)(G)--\nCurrent allocation mode is global\nLast OS error: No such file or directory\nCurrent file position is 18949\n'

    def __init__(self, msg: str, stderr: bytes = None, stdout: bytes = None) -> None:
        if stderr:
>           msg += '\n[stderr]\n' + stderr.decode(sys.getdefaultencoding(), 'replace')
E           AttributeError: 'str' object has no attribute 'decode'

sphinx/ext/imgmath.py:48: AttributeError
------------------------------------------------------------------------- Captured stdout teardown -------------------------------------------------------------------------
# testroot: root
# builder: html
# srcdir: /tmp/pytest-of-tkloczko/pytest-16/ext-math-simple
# outdir: /tmp/pytest-of-tkloczko/pytest-16/ext-math-simple/_build/html
# status:
Running Sphinx v4.3.0
loading pickled environment... done
building [html]: all source files
updating environment: 0 added, 0 changed, 0 removed
looking for now-outdated files... none found
preparing documents... done
writing output... [100%] index
# warning:

============================================================================= warnings summary =============================================================================
tests/test_build_linkcheck.py::test_invalid_ssl
  /home/tkloczko/rpmbuild/BUILD/Sphinx-4.3.0/sphinx/builders/linkcheck.py:490: ResourceWarning: unclosed <eventlet.green.ssl.GreenSSLSocket fd=41, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('127.0.0.1', 47566), raddr=('127.0.0.1', 7777)>
    return 'broken', str(err), 0

tests/test_build_linkcheck.py::test_invalid_ssl
  /home/tkloczko/rpmbuild/BUILD/Sphinx-4.3.0/sphinx/builders/linkcheck.py:490: ResourceWarning: unclosed <eventlet.green.ssl.GreenSSLSocket fd=17, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('127.0.0.1', 47564), raddr=('127.0.0.1', 7777)>
    return 'broken', str(err), 0

tests/test_build_linkcheck.py::test_connect_to_selfsigned_fails
  /home/tkloczko/rpmbuild/BUILD/Sphinx-4.3.0/sphinx/builders/linkcheck.py:490: ResourceWarning: unclosed <eventlet.green.ssl.GreenSSLSocket fd=43, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('127.0.0.1', 47570), raddr=('127.0.0.1', 7777)>
    return 'broken', str(err), 0

tests/test_build_linkcheck.py::test_connect_to_selfsigned_fails
  /home/tkloczko/rpmbuild/BUILD/Sphinx-4.3.0/sphinx/builders/linkcheck.py:490: ResourceWarning: unclosed <eventlet.green.ssl.GreenSSLSocket fd=17, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('127.0.0.1', 47568), raddr=('127.0.0.1', 7777)>
    return 'broken', str(err), 0

tests/test_theming.py::test_theme_api
tests/test_theming.py::test_theme_api
tests/test_theming.py::test_theme_api
tests/test_theming.py::test_theme_api
  /home/tkloczko/rpmbuild/BUILD/Sphinx-4.3.0/sphinx/application.py:1288: RemovedInSphinx60Warning: app.html_themes is deprecated.
    warnings.warn('app.html_themes is deprecated.',

-- Docs: https://docs.pytest.org/en/stable/warnings.html
========================================================================= short test summary info ==========================================================================
SKIPPED [1] tests/test_build_epub.py:378: Skipped because DO_EPUBCHECK is not set
SKIPPED [1] tests/test_ext_autodoc.py:1988: py39+ is required.
SKIPPED [1] tests/test_util_typing.py:152: python 3.9+ is required.
SKIPPED [1] tests/test_util_typing.py:162: python 3.10+ is required.
SKIPPED [1] tests/test_util_typing.py:204: python 3.9+ is required.
SKIPPED [1] tests/test_util_typing.py:216: python 3.9+ is required.
SKIPPED [1] tests/test_util_typing.py:295: python 3.10+ is required.
FAILED tests/test_ext_math.py::test_imgmath_svg - AttributeError: 'str' object has no attribute 'decode'
==================================================== 1 failed, 1699 passed, 7 skipped, 8 warnings in 358.18s (0:05:58) =====================================================

OS

Linux/x86_64

Python version

3.8.12

Sphinx version

4.3.0

Sphinx extensions

N/A

Extra tools

N/A

Additional context

N/A

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions