Trying to understand this bug that occurs in both Jupyter Notebook and Jupyter Lab in Python >= 3.8, I noticed that it is related to the async engine of IPython. Code that should execute synchronously is often detected as async:

In [1]: get_ipython().should_run_async('[x for x in l]')                        
Out[1]: True

It occurs for list comprehensions, set comprehensions, dict comprehensions and generator expressions, but it does not occur for these elements when they are inside a function definition (even when it is called):

In [2]: get_ipython().should_run_async('(lambda: [x for x in l])()')                        
Out[2]: False

I also checked other ast elements that make this function return True and the only ones that do are the expected ones: async for, async with, and await.

(Fixing this function may solve the bug that I reported on the jupyter/notebook repository, but I think there is something else going on with async executions: when I was developing a Jupyter Lab extension, one of my callbacks for kernel.requestExecute received the async response of a notebook cell. Even in an async execution, I think the original cell should receive the response and update its result)

Other information:

python -c "import IPython; print(IPython.sys_info())"
{'commit_hash': 'bb015d48b',
 'commit_source': 'repository',
 'default_encoding': 'UTF-8',
 'ipython_path': '/home/joao/projects/ipython/IPython',
 'ipython_version': '8.0.0.dev',
 'os_name': 'posix',
 'platform': 'Linux-4.15.0-20-generic-x86_64-with-debian-buster-sid',
 'sys_executable': '/home/joao/anaconda3/bin/python',
 'sys_platform': 'linux',
 'sys_version': '3.7.3 (default, Mar 27 2019, 22:11:17) \n[GCC 7.3.0]'}