DOC: Improve cross-links in thread safety documentation#30373
DOC: Improve cross-links in thread safety documentation#30373ngoldbaum merged 2 commits intonumpy:mainfrom
Conversation
|
Maybe tag as free-threading ? |
|
sure, done |
mattip
left a comment
There was a problem hiding this comment.
Thanks for thinking about the documentation. In general cross-linking to the same term should be limited to a single reference per page, since it is visually disturbing.
In particular I'd like to use this page as an example of good top-level documentation
Cool!
This also does a number of updates to the misc.rst file
Hah, surprised you found it at all. That page is really an orphan that probably should have been removed at some stage. I would prefer to not add to it, rather to remove from it and move the content elsewhere.
| not release the GIL. | ||
| not release the :term:`python:GIL`. | ||
|
|
||
| Thread-local state |
There was a problem hiding this comment.
errstate is stored in a contextvar, so it's actually context-local state. It's async-safe and thread-safe.
Maybe you could call it Context-local state? I'm not sure if there's a term for it that asyncio or CPython more generally uses.
Also the printoptions are a contextvar as well.
There was a problem hiding this comment.
The custom allocator object is also context-local: https://numpy.org/devdocs/reference/c-api/data_memory.html#configurable-memory-routines-in-numpy-nep-49
Carreau
left a comment
There was a problem hiding this comment.
Some suggestions, in particular I'd like your input as to wether put in See Also, or in Note the ref from the functions docstrings, or just not put it at all.
I'm not a huge fan of See Also, as there is no way reading it to know it's about concurency; and same with saying "Concurency Note", if you search for "Thread" it does not match, so one may miss the text.
| Note that integer divide-by-zero is handled by the same machinery. | ||
|
|
||
| The error handling mode can be configured using :func:`numpy.seterr`, | ||
| :func:`numpy.geterr`, :func:`numpy.seterrcall`, and the :func:`numpy.errstate` |
There was a problem hiding this comment.
I wouldn't mention geterr, seterr, and seterrcall here. Or if you feel like they have to be mentioned, I'd emphasize that errstate is the modern interface and the others should be avoided because context managers are easier to understand and there's no reason for users to implement their own.
There was a problem hiding this comment.
I don't think the have to be mentioned, there are 2 tensions here:
- do we address users of code that already contain those, in which case it maybe we should.
- Or do we target new uses that should not even be aware of seterr and co. so that it can be removed. I think we are more in the second case.
| NumPy maintains some state for ufuncs on a per-thread basis, which means each | ||
| thread/task has its own independent configuration of the `numpy.errstate` (see | ||
| :doc:`/reference/routines.err`), and of | ||
| :ref:`text_formatting_options`. |
There was a problem hiding this comment.
I'd add the following sentence to the end of this paragraph:
You can update state stored in a context variable by entering a context manager. As soon as the context manager exits, the state will be reset to its value before entering the context manager.
There was a problem hiding this comment.
I don't completely agree we need that; but I don't think it harms to add it.
I read this as "if you set it in one thread/task, it will only affects this thread/task" (which is either good or bad depending on what you want to achieve), the fact that you can use a context manager is independent of that.
Said in a different way, we state things about spacial-locality1 (thread/task), vs time-locality (context-manager)
Footnotes
-
Yes, I know spacial locality is more about where the data is, 'execution context' if you prefer, but I want to avoid 1 more use of "context", and yes threads may interleave in time as well, but I don't think users think about threads this way. ↩
Those are mostly cosmetic changes and extra cross-links to improve the documentation. In particular I'd like to use this page as an example of good top-level documentation for threading/freethreading in the py-threading-guide https://py-free-threading.github.io/ This also does a number of updates to the misc.rst file which was basically too indented, and missing a bunch of links. I think many of the info misc.rst on how to interop with C/C++ could be removed, and likely all the information on seterr/geterr moved to routines.err.rst but I don't want to include this moves as part of this PR as it's likely more subject to discussion. There are minor updates in conf.py to avoid 1 sphinx warnings, and I know the thread safety backlink in geterr/seterr/...etc, are a bit overkill, but as numpy will be used as an example and in general is referred to as one of the core scientific Python project I thinks it's OK go go a bit further than usual. [skip actions] [skip cirrus]
|
Squashed & rebased, sorry for the loss of history bur fixing each commit conflict with #30394 was annoying. |
|
I think I got everything. |
* DOC: Improve cross-links in thread safety documentation Those are mostly cosmetic changes and extra cross-links to improve the documentation. In particular I'd like to use this page as an example of good top-level documentation for threading/freethreading in the py-threading-guide https://py-free-threading.github.io/ This also does a number of updates to the misc.rst file which was basically too indented, and missing a bunch of links. I think many of the info misc.rst on how to interop with C/C++ could be removed, and likely all the information on seterr/geterr moved to routines.err.rst but I don't want to include this moves as part of this PR as it's likely more subject to discussion. There are minor updates in conf.py to avoid 1 sphinx warnings, and I know the thread safety backlink in geterr/seterr/...etc, are a bit overkill, but as numpy will be used as an example and in general is referred to as one of the core scientific Python project I thinks it's OK go go a bit further than usual. [skip actions] [skip cirrus] * typo
DOC: Improve cross-links in thread safety documentation (#30373)
This is similar to `numpy/numpy#30373`, and adds top level documentation as well as back reference to give example on how to use it with Python free threading.
This is similar to `numpy/numpy#30373`, and adds top level documentation as well as back reference to give example on how to use it with Python free threading. This is current still in draft, but I'd like to have it with a PR number while working on it and get feedback from interested person; Also my understanding of all those function and compiled code is limited so corrections welcomed. In the long run I'd like to point to this as a _good_ example of what to do from the py-free-threading guide.
This is similar to `numpy/numpy#30373`, and adds top level documentation as well as back reference to give example on how to use it with Python free threading. This is current still in draft, but I'd like to have it with a PR number while working on it and get feedback from interested person; Also my understanding of all those function and compiled code is limited so corrections welcomed. In the long run I'd like to point to this as a _good_ example of what to do from the py-free-threading guide. [docs only]
Those are mostly cosmetic changes and extra cross-links to improve the documentation. In particular I'd like to use this page as an example of good top-level documentation for threading/freethreading in the py-threading-guide https://py-free-threading.github.io/
This also does a number of updates to the misc.rst file which was basically too indented, and missing a bunch of links. I think many of the info misc.rst on how to interop with C/C++ could be removed, and likely all the information on seterr/geterr moved to routines.err.rst but I don't want to include this moves as part of this PR as it's likely more subject to discussion.
There are minor updates in conf.py to avoid 1 sphinx warnings, and I know the thread safety backlink in geterr/seterr/...etc, are a bit overkill, but as numpy will be used as an example and in general is referred to as one of the core scientific Python project I thinks it's OK go go a bit further than usual.
[skip actions]
[skip cirrus]