Background

This is another part of introducing pants, as discussed in various TSC meetings.

Related PRs can be found in:

• pantsbuild milestone (which includes PRs since Initialize pantsbuild #5713)
• pantsbuild label (which also includes preparatory PRs that came before adding pantsbuild)

Overview of this PR

This configures pants and generates a lockfile for a new pants-plugins/ directory.

In follow-up PRs I will add several plugins for pants to improve Developer Experience and simplify config we need to add in a lot of directories. This PR adds the prerequisite global config we need before introducing any of those plugins.

This adjusts config in two files: pants.toml and pants-plugins/BUILD.

Relevant Pants documentation

• Writing Plugins | Plugins overview
  • in repo plugins
  • Building in-repo plugins with Pants
• Targets:
  • pants_requirements

pants plugins

The pants plugin framework is very flexible. The core engine of pants is written in rust (for speed), but the plugin API is python. Each of the backends we have enabled, black, flake8, etc, are implemented as pants plugins. Plus, we can (and will) add in-repo plugins to handle some of the unique aspects of how StackStorm is organized.

By convention in-repo plugins go in the pants-plugins/ directory.

pants-plugins have a dependency on pants itself

At risk of stating the obvious, all of our in-repo pants-plugins (once added, will) depend on pants. We could add a python_requirement target to capture that dependency, but then updating pants would require updating the same version number in 2 places. Luckily pants provides some tools to help here.

In pants.toml, we register the pants.backend.plugin_development backend to get access to the pants_requirements target.

And then use pants_requirements in pants-plugins/BuILD:

Configure pants to find plugins in pants-plugins/

First we add pants-plugins to [GLOBAL].pythonpath. This pythonpath only refers to the path pants uses when loading itself, not our code or the other tools we use.

And we register another source root, so we can run black, flake8 and friends on our pants-plugins/ code:

pants-plugins resolve and lockfile

When we add an in-repo plugins we should keep this warning from the docs in mind:

In-repo dependencies
In-repo plugin code should not depend on other in-repo code outside of the pants-plugins folder. The pants-plugins folder helps isolate plugins from regular code, which is necessary due to how Pants's startup sequence works.

To better isolate the pants-plugins code from the rest of our code, we create a new resolve for it.

This registers the resolve+lockfile in pants.toml:

And then this adds python interpreter constraints to that resolve. These constraints are for code that runs in pants, so we match the python versions that pants itself uses.

And we use __defaults__ to make sure all of the plugin code, including our pants_requirements target, are part of the pants-plugins resolve. aside: I also added skip_pylint=True like in #5837. Because we're using __defaults__ we do not have to merge #5837 before this PR. The setting will apply as soon as it is available.

And finally, generate lockfiles/pants-plugins.lock using ./pants generate-lockfiles --resolve=pants-plugins.

pants-plugins/README.md

I also added a skeleton readme to explain what goes in the pants-plugins/ directory.