Skip to content

Make module state local to a module instance (PEP-489/573) #2343

@scoder

Description

@scoder

In order to support PEP-489 (multi-step module initialisation and reinitialisation), the global state of modules must become local to a module instance. Initial support for PEP-489 is in PR #1919.

Further links:

  • PEP-573 about accessing module state from extension type methods
  • PEP-575 about refactoring CPython's C functions
  • Meta-PEP 576 about function type changes

Approach for Cython:

  1. Require users to explicitly enable this feature with a compiler directive, as Cython cannot safely determine that a module supports this (there might be global state in external C libraries or C function callbacks), and there will be a visible performance impact.

  2. Move all globals into a struct that gets stored in the CPython module state.

  3. Use context variables (PEP 567) to pass the module state into C functions (also across coroutine dispatches etc.).

  4. Copy all extension types to the heap to make them specific to a module instance and their methods aware of the module.

  5. Allow users to make specific module globals shared across modules with a new declaration.

Thus, the following module:

cdef int x = 8

cdef int func():
    n = x
    return n

def get():
    print(self.x)
    return func()

def change(n):
    global x
    self.x = n

would internally be implemented as follows:

cdef int x = 8

cdef int func():
    __state = ContextVars['__cython_this_module_state_1234']
    n = __state.x
    return n

def get():
    with context(__cython_this_module_state_1234=__module_c_state__):
        print(self.x)
        return func()

def change(n):
    with context(__cython_this_module_state_1234=__module_c_state__):
        global x
        self.x = n

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions