Main stuff I'm worried about:

Does there need to be a separate directory for different (supported) versions of Python? 2 vs 3, or 3.5 vs 3.6?

And: I'm genuinely worried that the overhead cost for CPython might be so big that most of it won't be usefully fuzzable. Not sure how to evaluate this stuff.

Does there need to be a separate directory for different (supported) versions of Python

You may technically do either way.
It might be a bit cleaner to have separate dirs for major releases, but OTOH you will have more dockerfiles/etc to maintain. No strong opinion.
Also: will the different versions have the same set of maintainers?

the overhead cost for CPython might be so big that most of it won't be usefully fuzzable.

Can you explain?

(And this PR is good to go, once we decide on how to name it (cpython, or python3, or ...)

How much the overhead is going to be depends on how the fuzzers are written. I home if they're written as a C function which never drives the interpreter, just the C API for these modules, the overhead won't be too bad.

If your target function involves running the core python VM (ceval.c) though, you're going to have a really bad time.

I think python 2 and python 3 feel sufficiently different to be their own projects, you can rename cpython2 and cpython3. But i dont think we need to seperate 3.5 vs 3.6. We fuzz on trunk, so preferred to keep just 3.6. older version can go unmaintained and we don't want to fuzz and find old bugs.

OK, I split it in half to cpython2 and cpython3 in another commit.

Also: will the different versions have the same set of maintainers?

Yes! At least until python2 is retired. (EOL upstream is 2020 [at which point presumably someone else will take over maintaining it], EOL at Google is unknown to me.)

How much the overhead is going to be depends on how the fuzzers are written. I home if they're written as a C function which never drives the interpreter, just the C API for these modules, the overhead won't be too bad.

Yeah. For example, to fuzz hashing, we can do something like PyObject_Hash(PyString_FromStringAndSize(data, n)) (or maybe a little better than that). It still involves manipulating interpreter object and allocations and all that, but not dispatching on Python bytecode (probably...)

OTOH for some modules my initial drafts did involve running the interpreter. For example, the json/simplejson module does not export its C functions anywhere, so the draft I have on my machine just imports simplejson and calls simplejson.loads(some_string) from C -- which executes Python code (going through ceval.c), which then calls the C functions. If that's too slow, we'll have to change the API to export the underlying parsing functions (they are often static and effectively only visible to Python, without shenanigans).

I don't think that should be a problem, though, since upstream is interested in fuzzing as far as I can tell. We'll find a way to make it work and get decent tests out of it. :)