Allow ./x test to run tests without doc tests and without benchmarks#153143
Allow ./x test to run tests without doc tests and without benchmarks#153143Urhengulas wants to merge 6 commits intorust-lang:mainfrom
./x test to run tests without doc tests and without benchmarks#153143Conversation
|
The rustc-dev-guide subtree was changed. If this PR only touches the dev guide consider submitting a PR directly to rust-lang/rustc-dev-guide otherwise thank you for updating the dev guide with your changes. |
|
r? @jieyouxu rustbot has assigned @jieyouxu. Use Why was this reviewer chosen?The reviewer was selected based on:
|
772d7dc to
52327a7
Compare
|
I am pretty sure that following workaround will also become unnecessary when running rust/library/alloctests/benches/vec_deque_append.rs Lines 9 to 13 in 3a70d03 |
This is to keep the behaviour the same.
52327a7 to
5bd3628
Compare
|
@rustbot reroll |
|
r=me with commits squashed a bit (or fully) |
| /// | ||
| /// By default all build output will be placed in the current directory. | ||
| pub fn new(mut config: Config) -> Build { | ||
| if config.cmd.no_doc() { |
There was a problem hiding this comment.
If the code panics, then it's not really a deprecation :) Let's just print a warning, and move this to config parsing from here.
|
Reminder, once the PR becomes ready for a review, use |
Problem
For Ferrocene we would like to run only the
coretestsandallocteststest suites when measuring code coverage. Runningcorebenchesandallocbencheswould alter the numbers, which is not compliant with the certification.This is currently not possible in bootstrap. By default
./x testruns unit, integration and doc tests../x test --doconly runs doc tests. So far, so good.The problem is that while
./x test --no-docstops bootstrap from executing doc tests, it surprisingly starts executing benchmarks (next to examples and bins as well). It basically behaves likecargo test --all-targets.Solution
This PR renames the existing
--no-docflag to--all-targetsand keeps the same behaviour as before. Unfortunately it is not possible to internally switch fromcargo --bins --examples --tests --benchestocargo --all-targetsbecause it will fail e.g../x test library/alloc --all-targetswith an error like "use of unstable library featuretest".Additionally, this PR add a
./x test --testsflag (equivalent tocargo test --tests) that only executes unit and integration tests.Both changes we are doing in https://github.com/ferrocene/ferrocene anyways, but believe that they may be useful for other people as well and therefore would like to contribute them.
Note that this PR does not change the behaviour of either
./x testor./x test --doc.Note on
test = trueWhile this change enables bootstrap to run tests without doc tests and without benchmarks, executing
./x test library/core library/alloc --testswill still build and executecorebenchesandallocbenches.What?! 😱 Why all of this effort to enable it then?
Good question! The reason they are still executed is, that they are marked with
test = truein their respectiveCargo.toml(corebenches, allocbenches).@bjorn3 mentioned that it is important for upstream that those benchmarks are executed by default, even if no
./x --all-targetsis passed. This is perfectly possible with this PR. Benchmarks marked withtest = truewill be executed when calling either of./x test,./x test --testsor./x --all-targets. Therefore this PR does not include a commit to change that. We will just do this change in https://github.com/ferrocene/ferrocene.Open questions
Error message
I added one commit that adds an error message if a user passes
--no-docand points them to--all-targetsand--tests. I think it might be nice to have, but if you think it is not necessary, I can remove it.How to test this change
You can see the change in action by running
./x test library/alloc --testsand./x test library/alloc --all-targets. The first will executealloctestsandallocbenches(which is marked withtest = true), while the second will additionally runbenches/vec_deque_append.rs(which is not marked withtest = true).