This is the public tracking issue for the effort to port the Python rules from Java to Starlark.

I'll provide status updates here upon request or when there are particularly notable developments.

Internally, there's been a lot of progress on this. The work has stayed internal so far simply because, internally, Google has quite a few modifications to the rules and I am not/wasn't entirely sure how feasible things would be in practice, and I simply don't have time to, or experience in, managing the Google->Github side of things. Things have gone pretty good overall, though; we're very close to enabling py_binary and py_test within Google.

The rough plan is to re-implement the rules using the "builtins" mechanism of Bazel. This basically allows Bazel to bundle Starlark code with itself and provides mechanisms for Java<->Starlark interop, in particular ways for Starlark to call private APIs the Bazel team isn't sure should be generally public or not. While this means you would still have to wait for Bazel release to get changes, modifications should be much easier and accessible.

When and how to transition out of Starlark builtins to e.g. rules_python, is still TBD. That's probably going to depend on behavioral differences and the extent of reliance on the Java<->Starlark builtins interop.

Status (March 2023)

• All the rules are using the Starlark implementation
• Porting providers is in-progress

Within Google, we have the Starlark implementation enabled. So rewriting in Starlark is possible and works; just a matter of tracking down edge cases now.

Blockers

• None at the moment

Incompatibilities (or other observable changes in behavior)

• No python 2 support: as part of Bazel 7, support for Python 2 is being removed.
  This applies to the Java-implemented rules, too.
• --experimental_build_transitive_python_runfiles=true not supported: This is
  being dropped as part of Bazel 7 and applies to the Java-implemented rules, too.
  This feature enables an obscure behavior where building one target materializes
  the runfiles of dependent binaries as if they were directly built.
• stamp attribute can no longer be a configurable (select-based) boolean value.
  Integers (select-based or not) are still supported (and preferred).
• legacy_create_init attribute can no longer be a configurable (select-based boolean
  value. Integers (select-based or not) are still supported (and preferred).
• In cquery commands, the provider name as returned by the providers() function
  has changed. These "new" names are considered internal and should not be relied upon;
  they will change again when the rules are removed from Bazel and moved to the
  rule_python repository.
  • PyInfo -> @_builtins//:common/python/providers.bzl%PyInfo
  • PyRuntimeInfo -> @_builtins//:common/python/providers.bzl%PyRuntimeInfo
• For py_test to add the requires-darwin execution requirement, the Apple
  OS constraints (@platforms//os:{ios,macos,tvos,watchos}) must match. Setting
  --cpu flag (e.g --cpu=darwin_x86_64) isn't sufficient. Note that this only
  affects remotely running a py_test built for Apple platforms.

TODO

• Implement imports attribute
• PyInfo provider
• PyRuntimeInfo provider
• PyCcLinkParamsProvider provider
• Port flags to Starlark
• py_binary / py_test
• py_library
• py_runtime
• Re-enable allowing using the Starlark impl for tests where it was disabled for Google-internal reasons (e.g. imports attribute)
• Bazel regression tests passing
• Python 2 disabled
• experimental_build_transitive_python_runfiles disabled
• Enable the rules
• Enable the providers
• Fix downstream project failures

cc: @comius @brandjon