Skip to content

Commit 7bec7b7

Browse files
committed
[mypyc] Document librt and librt.base64 (#21114)
This also contains a few small unrelated changes to mypyc docs.
1 parent c482596 commit 7bec7b7

File tree

5 files changed

+144
-4
lines changed

5 files changed

+144
-4
lines changed

mypyc/doc/index.rst

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,13 @@ generate fast code.
2727
differences_from_python
2828
compilation_units
2929

30+
.. toctree::
31+
:maxdepth: 2
32+
:caption: librt: Runtime Library reference
33+
34+
librt
35+
librt_base64
36+
3037
.. toctree::
3138
:maxdepth: 2
3239
:caption: Native operations reference

mypyc/doc/introduction.rst

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -126,7 +126,10 @@ Mypyc uses several techniques to produce fast code:
126126
Value types only need to be checked in the boundaries between
127127
dynamic and static typing.
128128

129-
* Compiled code uses optimized, type-specific primitives.
129+
* Compiled code uses optimized, type-specific primitives for many
130+
Python standard library features. The :ref:`librt <librt>` package
131+
also provides optimized alternatives for certain standard library
132+
features.
130133

131134
* Mypyc uses *early binding* to resolve called functions and name
132135
references at compile time. Mypyc avoids many dynamic namespace

mypyc/doc/librt.rst

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
.. _librt:
2+
3+
Librt overview
4+
==============
5+
6+
The `librt <https://pypi.org/project/librt/>`_ package defines fast
7+
primitive operations that are optimized for code compiled
8+
using mypyc. It has carefully selected efficient alternatives for
9+
certain Python standard library features.
10+
11+
``librt`` is a small, focused library. The goal is not to reimplement
12+
the Python standard library, but to address specific gaps or
13+
bottlenecks.
14+
15+
Librt contents
16+
--------------
17+
18+
Follow submodule links in the table to a detailed description of each submodule.
19+
20+
.. list-table::
21+
:header-rows: 1
22+
:widths: 30 70
23+
:width: 100%
24+
25+
* - Module
26+
- Description
27+
* - :doc:`librt.base64 <librt_base64>`
28+
- Fast Base64 encoding and decoding
29+
30+
Installing librt
31+
----------------
32+
33+
When you install mypy, it will also install a compatible version of librt as a
34+
dependency. If you distribute compiled wheels or install compiled modules in
35+
environments without mypy installed, install librt explicitly or depend on it
36+
with a version constraint (but it's only needed if your code explicitly imports
37+
``librt``), e.g. ``python -m pip install librt>=X.Y``.
38+
39+
If you don't have a recent enough librt installed, importing librt will fail.
40+
Compiled code often needs a version of librt that is not much older than the
41+
mypyc being used.
42+
43+
Backward compatibility
44+
----------------------
45+
46+
We aim to keep librt backward compatible. It's recommended that you allow users
47+
of your published projects that use librt to update to a more recent version. For
48+
example, use a ``>=`` version constraint in your ``requirements.txt``.
49+
50+
Using librt in non-compiled code
51+
--------------------------------
52+
53+
Using librt in code that is *not* compiled with mypyc is fully supported. However,
54+
some librt features may have significantly degraded performance when used from
55+
interpreted code. We will document the most notable such cases, but it's always
56+
recommended to measure the performance impact when considering a switch from Python
57+
standard library functionality to librt in a non-compiled use case.

mypyc/doc/librt_base64.rst

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
.. _librt-base64:
2+
3+
librt.base64
4+
============
5+
6+
The ``librt.base64`` module is part of the ``librt`` package on PyPI, and it includes
7+
base64 encoding and decoding functions that use SIMD (Single Instruction, Multiple Data)
8+
for high efficiency. It is a wrapper around
9+
`Alfred Klomp's base64 library <https://github.com/aklomp/base64>`_.
10+
11+
These functions are mostly compatible with the corresponding functions in the
12+
Python standard library ``base64`` module but are significantly faster,
13+
especially in code compiled with mypyc. For larger inputs, these are much
14+
faster than the standard library alternatives even when used from interpreted code.
15+
16+
.. note::
17+
18+
The decode functions don't behave identically when data is malformed.
19+
Only commonly used functionality is provided. The supported arguments may be
20+
restricted compared to the stdlib functions: the optional ``altchars`` and
21+
``validate`` arguments are not supported.
22+
23+
Functions
24+
---------
25+
26+
.. function:: b64encode(s: bytes) -> bytes
27+
28+
Encode a bytes object using Base64 and return the encoded bytes.
29+
30+
This is equivalent to ``base64.b64encode(s)`` in the standard library.
31+
32+
.. function:: b64decode(s: bytes | str) -> bytes
33+
34+
Decode a Base64 encoded bytes object or ASCII string and return
35+
the decoded bytes.
36+
37+
Non-base64-alphabet characters are ignored. This is compatible
38+
with the default behavior of standard library ``base64.b64decode``.
39+
40+
Raise ``ValueError`` if the padding is incorrect. The standard
41+
library raises ``binascii.Error`` (a subclass of ``ValueError``)
42+
instead.
43+
44+
.. function:: urlsafe_b64encode(s: bytes) -> bytes
45+
46+
Encode a bytes object using the URL and filesystem safe Base64
47+
alphabet (using ``-`` instead of ``+`` and ``_`` instead of ``/``),
48+
and return the encoded bytes.
49+
50+
This is equivalent to ``base64.urlsafe_b64encode(s)`` in the
51+
standard library.
52+
53+
.. function:: urlsafe_b64decode(s: bytes | str) -> bytes
54+
55+
Decode a bytes object or ASCII string using the URL and filesystem
56+
safe Base64 alphabet (using ``-`` instead of ``+`` and ``_`` instead
57+
of ``/``), and return the decoded bytes.
58+
59+
This is an alternative to ``base64.urlsafe_b64decode(s)`` in the
60+
standard library.
61+
62+
Raise ``ValueError`` if the padding is incorrect. The standard
63+
library raises ``binascii.Error`` (a subclass of ``ValueError``)
64+
instead.

mypyc/doc/performance_tips_and_tricks.rst

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,11 @@ perhaps easily reimplement them in type-annotated Python, or extract
3636
the relevant code and annotate it. Now it may be easy to compile this
3737
code to speed it up.
3838

39-
Second, you may be able to avoid the library altogether, or use an
39+
Second, the :doc:`librt <librt>` package provides optimized
40+
alternatives for certain standard library features that are designed
41+
for mypyc-compiled code.
42+
43+
Third, you may be able to avoid the library altogether, or use an
4044
alternative, more efficient library to achieve the same purpose.
4145

4246
Type annotations
@@ -155,9 +159,9 @@ Here are examples of features that are fast, in no particular order
155159
* Calling methods of native classes defined in the same compilation
156160
unit (with positional and/or keyword arguments)
157161

158-
* Many integer operations
162+
* Many :ref:`integer operations <int-ops>`
159163

160-
* Many ``float`` operations
164+
* Many :ref:`float operations <float-ops>`
161165

162166
* Booleans
163167

@@ -181,6 +185,9 @@ Here are examples of features that are fast, in no particular order
181185

182186
* Comparing strings for equality
183187

188+
* All features in the :doc:`librt <librt>` package, which are optimized
189+
for compiled code
190+
184191
These features are also fast, but somewhat less so (relative to other
185192
related operations):
186193

@@ -192,6 +199,8 @@ related operations):
192199

193200
* Native :ref:`dict <dict-ops>` and :ref:`set <set-ops>` operations
194201

202+
* Native :ref:`str <str-ops>` and :ref:`bytes <bytes-ops>` operations
203+
195204
* Accessing module-level variables
196205

197206
Generally anything documented as a native operation is fast, even if

0 commit comments

Comments
 (0)