Skip to content

Add a conftest target type#10613

Closed
Eric-Arellano wants to merge 1 commit intopantsbuild:masterfrom
Eric-Arellano:conftest
Closed

Add a conftest target type#10613
Eric-Arellano wants to merge 1 commit intopantsbuild:masterfrom
Eric-Arellano:conftest

Conversation

@Eric-Arellano
Copy link
Copy Markdown
Contributor

Problem

Some Python repositories (like ours) colocate source files with tests.

In #10603, we stopped having conftest.py default to a python_tests target and to instead default to python_library. This was a good conceptual change, and also necessary now that we always run tests one-file-at-a-time.

However, #10603 does not work well in repositories where tests are colocated with source files. Previously, the python_library would solely refer to production code, but now it's including a test config file without users realizing it. This means, for example, that test's inferred dependencies will be included in the python_library(), so, whenever a user depends on :library (rather than file dep), they'll pull in more than they expected.

For example, in Toolchain's Django repo, we have to explicitly declare several dynamic dependencies for our conftest.py. To avoid polluting the production targets, this means that we need to declare this in our BUILD files:

python_library(
  sources=["*.py", "!conftest.py", "!*_test.py"],
  dependencies=[
     ...  
  ],
)

python_library(
  name="conftest",
  sources=["conftest.py"],
  dependencies=[
    ...
  ],
)

Beyond being clunky, this is subtle. It will be very easy for us to forget to declare two separate targets and to not use default source globs.

Solution

Leverage the Target API's design to create a new conftest() target. This behaves like a python_library(), and is basically sugar for python_library(sources=["conftest.py"]).

Update python_library() to exclude conftest.py by default, so that it doesn't mix production and test code.

Note that a macro could solve the problem for an individual codebase, but we want to solve this problem for all our users.

Result

Now, Toolchain's BUILD file could look like this:

python_library(
  dependencies=[
     ...  
  ],
)

conftest(
  name="conftest",
  dependencies=[
    ...
  ],
)

A user can still use a python_library() target for their conftest.py, of course. They will still use python_library() for test utils in general. This is about our default globs doing the right thing.

[ci skip-rust]
[ci skip-build-wheels]

# Rust tests and lints will be skipped. Delete if not intended.
[ci skip-rust]

# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
@coveralls
Copy link
Copy Markdown

Coverage Status

Coverage remained the same at 0.0% when pulling c65e843 on Eric-Arellano:conftest into ed40678 on pantsbuild:master.

Copy link
Copy Markdown
Contributor

@benjyw benjyw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@cosmicexplorer
Copy link
Copy Markdown
Contributor

Note that a macro could solve the problem for an individual codebase,

I'm not sure I follow this logic. Especially since we are describing this change as "basically sugar", it feels to me like a macro would map more specifically to my mental model of this. Do we expect to be writing rules at some point that have to specifically process conftest() declarations differently than others?

@Eric-Arellano
Copy link
Copy Markdown
Contributor Author

it feels to me like a macro would map more specifically to my mental model of this

The issue is that a macro defined in pantsbuild/pants by definition is locally scoped - it won't be exported to end-users. A downstream repo would need to copy that macro definition, and then they would also need to be diligent to use my_python_library instead of python_library to make sure it doesn't subtly and inadvertently own the conftest.py file.

Instead, here, we make sure that python_library doesn't ever do this subtle thing of mixing test config with production files, unless the user explicitly does that by setting sources.

We will also be able to document in https://www.pantsbuild.org/v2.0/docs/python-test-goal about using a conftest() target if you have a conftest.py. That will be nice because conftest.py already has special logic via --python-infer-confests. Rather than having to explain how we will "generate a subtarget" from the owning python_library target, we can explain that you should create a conftest.py target, and then Pants will infer a dependency on all ancestor conftest.py files. I think this will be simpler to understand.

Do we expect to be writing rules at some point that have to specifically process conftest() declarations differently than others?

Unlikely. But that was an explicit design goal for the Target API. You can create new target types that fit your domain and still have those be understood by Pants with zero changes. For example, some repos might want to create a django_library target type.

@Eric-Arellano
Copy link
Copy Markdown
Contributor Author

This goes too far in favor of purity over ergonomics. Closing in favor of #10619. Turns out, conftest.py has more guarantees that we originally thought, and it's fairly safe for it to belong to python_tests.

@Eric-Arellano Eric-Arellano deleted the conftest branch August 15, 2020 00:15
Eric-Arellano added a commit that referenced this pull request Aug 15, 2020
…10619)

In #10603, we moved `conftest.py` from `python_tests` to `python_library`. This is because Pants now runs file-at-a-time for Pytest thanks to the model change in #10511. This causes `conftest.py` to error because it doesn't have any test files.

We decided in #10603 that `conftest.py` should be under `python_library` because it is not actual tests. For example, test util code goes under a `python_library` target already.

However, `python_library` owning a `conftest.py` by default ended up being problematic, as it caused test code to be mixed with production code, which is generally not desired. See #10613 for an approach to fix this.

Instead of adding a `conftest` target, this PR instead moves back `conftest.py` to `python_tests` and special cases `pytest_runner.py` to skip `conftest.py`. Even though this is less correct, it's simpler for users and it avoids making 1.x users having to change a bunch of things. Conceptually, `conftest.py` can be seen as "config" for tests, rather than traditional "test utils" like you'd have in a `python_library`.

[ci skip-rust]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants