Allow building rustdoc without first building rustc (MVP)#79540
Allow building rustdoc without first building rustc (MVP)#79540bors merged 2 commits intorust-lang:masterfrom
Conversation
|
Once you'be built with |
I discussed this in 'Unresolved Questions'.
Hmm, you mean have |
|
I think rustup management is the wrong approach; we should be able to use similar logic to LLVM downloads and get a rustc from previous CI commit. That'll always work if you don't modify rustc (which we can check via git). This would still require using x.py but I think that's actually not a huge burden in practice - it should work with any python version. I think we can do more to ease the introduction to x.py, but I would rather do so for all contributors rather than piecemeal. |
|
Ok, I like that idea - then most of the concerns here go away because you're still going through x.py, and |
|
That sounds like a great idea indeed! |
|
@Mark-Simulacrum ideally this would only download the rustc if you were going to use it for something (and not for, say |
|
I'm only adding remarks. Iknow that my understanding of the big picture is flawed, so feel definitively free to ignore any of those. And this is definitively not a rant, but pure feedback. I am certain that least when those where introduced they made total sense. I am not sure if this PR is the right place to discuss those points either. If you want to, it may be better to continue on zulip. And I really think that this PR goes in the right direction.
Regarding this very specific sub issue, I think that adding a
As an external contributor, I 100% understand the need for a tool like I think that having a dedicated Finally, rustdoc doesn't feel any more special than any other rust project from an exteral point of view. Since, the documentation for |
|
I recently found out that the downloads shell out to curl or whatever on platforms anyway, so I'm less concerned now about keeping the logic there, but I would like to avoid duplicating it if possible. I would not worry about extra downloads to start though, I expect almost everyone to need that download anyway. Note that it shouldn't be a problem to use that compiler for bootstrap as well, so you'd just be downloading a different set of libraries, not too much more (well rustc-dev would be new, but that's I think roughly 60MB overhead these days, so not horrible). |
|
@robinmoussu we can discuss more on Zulip if you like, but the tl;dr is that
I also want to caution against trying to fix every problem at once; that's a good way to fix none of them (as I found in rust-lang/rustc-dev-guide#843). |
|
Ok, this now downloads stage1 rustc from commit build artifacts if |
|
Oh and CI is failing because it uses |
|
Annoyingly, GitHub actions makes the .git directory read-only, so I can't run
|
This comment has been minimized.
This comment has been minimized.
9906e34 to
fe8bd9f
Compare
This comment has been minimized.
This comment has been minimized.
|
Okay - I've finally had some time to review this, sorry for the long wait. I think I am prepared to r+ this after a rebase to squash out the iterative fixes. Please also update the top level description to drop the strike outs - if you'd like to document future work, that's great, but a separate section is better. That said, I do have some concerns, but not ones that make me want to block this:
I also note you've (or someone) has marked this as needing FCP, but I do not think that is the case, so curious to hear more. |
I agree I don't think
Hmm, the idea is that you choose in config.toml exactly what components you want to build from source? That seems useful if e.g. you want to build clippy from source but not rustdoc, but I don't know how it would work for std since the compiler depends on the master version of libstd.
This sounds really awesome :D
Oh well, that probably isn't necessary. I think I had in my head "this should get feedback from the rustdoc team" but since then there have been a lot of hearts (which I assume means people are in favor :P) and I'm also now the team lead, so I don't know if we need to go throguh the formality of checkboxes. |
|
(sorry, I hate github's UI sometimes) |
src/bootstrap/bootstrap.py
Outdated
There was a problem hiding this comment.
This already went wrong :/ I just rebased over 9a1d617, which is a bors commit, but git doesn't recognize as modifying any code:
$ git show 9a1d6174c925f54c923599e29b09d6855e6b3a78
commit 9a1d6174c925f54c923599e29b09d6855e6b3a78 (origin/master, origin/HEAD, master)
Merge: 37d067f5d71 11e7897eae7
Author: bors <bors@rust-lang.org>
Date: Sat Feb 6 18:03:37 2021 +0000
Auto merge of #81832 - jonas-schievink:rollup-3nw53p0, r=jonas-schievink
Rollup of 7 pull requests
Successful merges:
- #81402 (tidy: Run tidy style against markdown files.)
- #81434 (BTree: fix documentation of unstable public members)
- #81680 (Refactor `PrimitiveTypeTable` for Clippy)
- #81737 (typeck: Emit structured suggestions for tuple struct syntax)
- #81738 (Miscellaneous small diagnostics cleanup)
- #81766 (Enable 'task list' markdown extension)
- #81812 (Add a test for escaping LLVMisms in inline asm)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
$
merge-base correctly says 9a1d617 was most recent change, but log --author=bors shows cfba499 instead, which is wrong, because compiler/ was modified since then (specifically in 7acf9ec, 85fb5cd, f631410, and 96e843c). In this case it didn't hurt anything, but if any of the APIs used by rustdoc had changed, it would have caused a build failure.
I really do think merge-base is more correct here. But I'm ok with fixing it in a follow-up if you like.
There was a problem hiding this comment.
It should be possible to pass -m1 or something to log to treat merge commits as modification; it shouldn't be a fundamental limitation. But maybe I need to read and understand merge base. Future PR, I think.
There was a problem hiding this comment.
I tried -m and it didn't help. But I agree this can wait for a follow-up :)
There was a problem hiding this comment.
Another alternative is to run git log --author=bors without adding compiler/ to the end, and then run git log <output of previous command> compiler/ on top of that. But that doesn't seem any simpler than merge-base.
|
This is ready to merge if you think it is :) I didn't change anything since the last review other than fixing conflicts and squashing the commits. If you like you can compare the changes with |
- Use the same compiler for stage0 and stage1. This should be fixed at some point (so bootstrap isn't constantly rebuilt). - Make sure `x.py build` and `x.py check` work. - Use `git merge-base` to determine the most recent commit to download. - Copy stage0 to the various sysroots in `Sysroot`, and delegate to Sysroot in Assemble. Leave all other code unchanged. - Rename date -> key This can also be a commit hash, so 'date' is no longer a good name. - Add the commented-out option to config.toml.example - Disable all steps by default when `download-rustc` is enabled Most steps don't make sense when downloading a compiler, because they'll be pre-built in the sysroot. Only enable the ones that might be useful, in particular Rustdoc and all `check` steps. At some point, this should probably enable other tools, but rustdoc is enough to test out `download-rustc`. - Don't print 'Skipping' twice in a row Bootstrap forcibly enables a dry run if it isn't already set, so previously it would print the message twice: ``` Skipping bootstrap::compile::Std because it is not enabled for `download-rustc` Skipping bootstrap::compile::Std because it is not enabled for `download-rustc` ``` Now it correctly only prints once. ## Future work - Add FIXME about supporting beta commits - Debug logging will never work. This should be fixed.
|
@bors r+ rollup=never I think this is good to go. I'm not sure if the ergonomics are quite right but we'll find out through experimentation :) |
|
📌 Commit 4aec8a5 has been approved by |
|
☀️ Test successful - checks-actions |
Motivation
The compile times for rustc are extremely long and a major issue for
recruiting new contributors to rustdoc. People interested in joining
often give up after running into issues with submodules or python
versions. stage1 rustdoc fundamentally doesn't care about bootstrapping
or stages, it just needs
rustc_privateavailable.Summary of Changes
[rust] download_rustcoptionlog --author=borsRustdocfor any stage. Instead, copy the CI artifacts from the downloaded sysroot stage0/ to stage0-sysroot/ or stage1/ inSysroot. This is done with anENABLE_DOWNLOAD_STAGE1constant which is off by default.The vast majority of work is done in bootstrap.py, which downloads the artifacts and extracts them to stage0/ in place of the beta compiler. Rustbuild just takes care of copying the artifacts to stage1 if necessary.
Future work
x.py setup toolswill compile rustbuild twice. Instead, this should download a separate beta compiler for stage0 and only use CI artifacts for stage1 onward. Allow building rustdoc without first building rustc (MVP) #79540 (comment)x.py setup toolsto enable this conveniently (it doesn't make sense to use this for compiler developers). jyn514@cb5d8c8debugandtracelogging). Allow building rustdoc without first building rustc (MVP) #79540 (comment), jyn514@6a5d512git log --author=borssometimes breaks. This should usegit merge-baseinstead. Allow building rustdoc without first building rustc (MVP) #79540 (comment)Some of this work has already been done in (the history for) jyn514@673476c.