bpo-40334: Catch E_EOF error when the tokenizer returns ERRORTOKEN #19743
Conversation
An E_EOF error was only being caught after the parser exited before this PR. There are some cases though, where the tokenizer returns ERRORTOKEN *and* has set an E_EOF error (like when EOF directly follows a line continuation character) which weren't correctly handled before.
The tag is still not here: https://github.com/python/cpython/releases but who knows ¯_(ツ)_/¯ |
Doesn't mean a thing -- I checked for that for something I wanted in the previous alpha, but IIUC the release managers first agree on a commit to build, then all build it locally, compare results, and finally push the tag upstream. |
Unfortunately so: https://github.com/ambv/cpython/tree/v3.9.0a6 |
Ohhhh, that makes sense. |
|
Yes, the point is: I put the tag on my tree, Steve and Ned build installers for their respective systems. It's not common but it did happen in the past that building those installers surfaced bugs and I had to move the tag. So I always wait with tagging until everything got cleanly built and tested (we also install the installers and run tests). |
Cool! Thanks for the explanation :) |
An E_EOF error was only being caught after the parser exited before
this PR. There are some cases though, where the tokenizer returns
ERRORTOKEN and has set an E_EOF error (like when EOF directly
follows a line continuation character) which weren't correctly
handled before.
https://bugs.python.org/issue40334