Skip to content

run_cell_async can't be re-entered without messing history #15087

@drorspei

Description

@drorspei

The function run_cell_async in interactiveshell.py, the function that actually executes user code, cannot be re-entered without messing with history causing a new session to be used. A simple command to run in ipython to see this is get_ipython().run_cell(raw_cell="None", store_history=True) . The reason this messes with the history is the order of instructions:

  1. store prompt in history at execution_count
  2. run the user code
  3. increment execution_count

Why should anyone care about this? Does anyone ever run run_cell_async when already running run_cell_async?? Why yes! Fixing this lets us modify frontends so that they can run multiple cells in parallel. For example, a tiny change to ipykernel makes it possible to run async cells concurrently. In fact, I've written a small one-file package that allows running async cells concurrently, as well as moving active cells to the background using a magic %bg - which you can run after a cell already started running.

https://github.com/drorspei/bgipykernel

The ability to run async cells in parallel is nice for those actually running async functions, such as web stuff. The ability to move a cell to the background has been very useful to me in the cases where I started running a cell that calls a foreign function that I realized would either never end, or worse - eat up all my RAM in the next minute. Moving the cell to the background lets the user quickly save state to disk, and then kill the kernel.

The fix to this issue is to change the order of instructions to:

  1. store prompt in history at execution_count
  2. store state of this execution count and use it in all displays
  3. increment execution_count
  4. run user code

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions