Track whether module declarations are inline (fixes #12590)#52319
Conversation
|
r? @pnkfelix (rust_highfive has picked a reviewer for you, use r? to override) |
|
The job Click to expand the log.I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact |
|
Woops, forgot to run tidy |
There was a problem hiding this comment.
This is a relatively minor point, but: You might as well move this line into the body of fn eval_src_mod, right? We might as well have it fully encapsulate setting up all the state related to reading a file as a module, rather than leaving this one step for its caller...
There was a problem hiding this comment.
Yes, I was afraid maybe someone would call eval_src_mod from somewhere else, but later I discovered this is the only place it is called.
|
A more important issue: I actually am in the camp that regards the current behavior as a feature, not a bug. I like getting a single file with the whole module tree in it... Having said that, I guess I'd be okay if the semantics of "one file with whole module tree" were restricted to Can you tell me off hand how your PR handles
|
|
Good point, I'll add a test for expanded to verify that it works for expanded. |
|
@pnkfelix I made the behavior as you described, but for some reason the test suite doesn't pick it up. I tried to debug by adding printlines to the parser, but that breaks compilation of stage0, and for some reason It's hard to keep a nice development pace when it's interspersed by these 15 minute+ compilation bouts, I spent 4 hours on finding out that my println!'s were breaking the stage0 compilation, just because I get distracted between invocations and go do other tasks. Do you have some trick to be productive while working on rustc? |
|
@tinco I'll check out your repo and take a look. Out of curiosity, in your attempt to instrument the code, why did you use |
|
@tinco I think I see what you mean: when I run I'll keep digging for a bit. Not sure whether this is some |
|
OH! The That's why, for example, if you add I don't even know if we specify how So, what now?! We can either:
|
|
I bet the reason that |
|
If you want to try to tackle changing rust/src/tools/compiletest/src/runtest.rs Line 393 in 025e04e The iterative loop I described is starts here: rust/src/tools/compiletest/src/runtest.rs Line 416 in 025e04e The actual invocations of the compiler are driven by the calls to rust/src/tools/compiletest/src/runtest.rs Line 424 in 025e04e ... whose source code is right below, here: rust/src/tools/compiletest/src/runtest.rs Line 498 in 025e04e and you can see the spot where we are feeding in rust/src/tools/compiletest/src/runtest.rs Line 503 in 025e04e So I might suggest changing |
|
In fact, after writing up those instructions above, I figured it would be easiest if I just went ahead and made the necessary change to your PR myself. |
|
@bors r+ |
|
📌 Commit fcbb3dd has been approved by |
|
hi @pnkfelix thanks a lot for chiming in on this PR! I would've probably taken a couple days to figure out how compiletest worked :) I used println! instead of debug! because I didn't know/forgot it existed :P I had a 2 year break from rust, and although I'm still ok with the general idea and semantics, I'm a bit rusty (hehe) with many workflow details. Working on these compiler bugs is fun because I read through loads of rust and get back into it :) I found this bug just by filtering a little bit and sorting on oldest first, and looking at the first one that didn't have many people theorizing on what to do. If you know of any other bug that's ready for someone to dive in to that has slightly more relevance to the real world than this one please let me know! |
|
⌛ Testing commit fcbb3dd with merge bc234f177bb2202a1a56d62c09dffb3f4737e0a0... |
|
💔 Test failed - status-appveyor |
…en they *have* to. This allows us to test expansion of files that use `mod foo;` syntax.
|
Ping from triage @pnkfelix: It looks like this PR is ready for your review. |
|
@bors r+ |
|
📌 Commit b985e91 has been approved by |
Track whether module declarations are inline (fixes #12590) To track whether module declarations are inline I added a field `inline: bool` to `ast::Mod`. The main use case is for pretty to know whether it should render the items associated with the module, but perhaps there are use cases for this information to not be forgotten in the AST.
|
yessss.... 🥁 |
|
☀️ Test successful - status-appveyor, status-travis |
|
Nice, this PR will make rustfmt's life a bit easier 🙌 |
Stabilize `#[my_macro] mod foo;` (part of `proc_macro_hygiene`) # Stabilization report ## Summary Allow `mod foo;` (an "outlined" module) in the input of attribute and derive (proc) macros. Tracking: - rust-lang#54727 - The `proc_macro_hygiene` feature covers multiple related features. This PR proposes only a partial stabilization Reference PRs: - rust-lang/reference#2284 ### What is stabilized > Describe each behavior being stabilized and give a short example of code that will now be accepted. Currently, `mod foo;` (an outlined / non-inline) module is explicitly rejected in the input of derive and attribute (proc) macros. This PR removes the check, allowing `mod foo;`. The module is provided to the macro as it is written in the source code, that is in its "not-yet-loaded" form without a body. ```rust // The following was previously rejected and will now compile. #[my_proc_macro] mod foo; #[my_proc_macro] fn bar() { #[path = "foo.rs"] mod foo; } #[derive(MyTrait)] struct Foo([u8; { #[path = "foo.rs"] mod foo; 0 }]); ``` ### What isn't stabilized > Describe any parts of the feature not being stabilized. Talk about what we might want to do later and what doors are being left open for that. If what we're not stabilizing might lead to surprises for users, talk about that in particular. The remaining parts of `proc_macro_hygiene` are not stabilized (specifically, attribute macros in places where they aren't currently allowed). ## Design ### Reference > What updates are needed to the Reference? Link to each PR. If the Reference is missing content needed for describing this feature, discuss that. rust-lang/reference#2284 ### RFC history > What RFCs have been accepted for this feature? This feature has not been explicitly specified by any RFC. - Ahead of the `proc_macro` stabilization, `#[my_macro] mod foo;` specifically was feature gated. - As a side effect of rust-lang#52319, the (unstable) behavior changed: - Before that PR, `mod foo;` was actually provided to macros as the loaded `mod foo { ... }` - After that PR, `mod foo;` was provided to macros as-is. - In 2019 (after the `proc_macro` stabilization, rust-lang#66078 made `mod foo` _**anywhere**_ in the input to derive or attribute proc macros unstable. - Before that PR, only proc macros directly applied to `mod foo;` were unstable. ### Answers to unresolved questions > What questions were left unresolved by the RFC? How have they been answered? Link to any relevant lang decisions. N/A ### Post-RFC changes > What other user-visible changes have occurred since the RFC was accepted? Describe both changes that the lang team accepted (and link to those decisions) as well as changes that are being presented to the team for the first time in this stabilization report. See the RFC history above. ### Key points > What decisions have been most difficult and what behaviors to be stabilized have proved most contentious? Summarize the major arguments on all sides and link to earlier documents and discussions. The main design decision is whether outlined modules should be passed to proc macros with or without their body. Arguments in favor of "without their body": - (To the best of my knowledge) the current compiler architecture makes "without their body" the natural choice. - To [quote](rust-lang#54727 (comment)) @petrochenkov, "it's clear that the current behavior of `#[my_macro] mod foo;` (the "not-yet-loaded" version of the module is passed to the macro) is natural to the current setup, stable and is not going to change". - At least I would be somewhat surprised to see a macro (which is operating on the AST) able to see or affect code outside the current file. I would assume by default that they get as input exactly what I have written. - Function-like procedural macros operate on arbitrary token trees. `mod foo;` in their input is not even recognized by the compiler and thus not expanded. Attribute and derive proc macros also receiving the unexpanded `mod foo;` in their input makes the two consistent. - Other syntactic constructs, e.g. other macros like `inlcude!("foo.rs")`, are also not expanded before being passed to proc macros. Providing modules without their body would be consistent with that. - Providing `mod foo;` to proc macros with its body would (as far as I know) be the only case where a proc macro would not get exactly what is written in the source code as its input. Arguments in favor of "with their body": - `#[my_macro] mod foo;` and `#[my_macro] mod foo {}` would have different behavior, which means that in the presence of proc macros, migrating an inline module to an outlined one would no longer be a purely syntactic change, but could affect the behavior of the proc macro. - `#[my_macro] mod foo;` and `#![my_macro]` in `foo.rs` ([currently unstable](rust-lang#54726)) would have different behavior. - Other attributes already have different behaviors depending on whether they are applied as inner vs outer attributes. For example `#[cfg(false)]` on a module as an outer attribute means the corresponding file can be missig or have syntax errors, while using it as an inner attribute requires the file to be syntactically valid. - Some macros may need access to (or change) the module body to function properly. ### Nightly extensions > Are there extensions to this feature that remain unstable? How do we know that we are not accidentally committing to those? No. ### Doors closed > What doors does this stabilization close for later changes to the language? E.g., does this stabilization make any other RFCs, lang experiments, or known in-flight proposals more difficult or impossible to do later? None. If, in the future, there is a desire to make the module body of outlined modules available to proc macros, there are multiple ways to add that in a backward-compatible way (e.g. an explicit opt-in during the proc macro registration, or an API in the `proc_macro` crate). ## Feedback ### Call for testing > Has a "call for testing" been done? If so, what feedback was received? No. ### Nightly use > Do any known nightly users use this feature? Counting instances of `#![feature(FEATURE_NAME)]` on GitHub with grep might be informative. Searching Github [produces](https://github.com/search?q=%22feature%28proc_macro_hygiene%29%22+language%3ARust+&type=code) 4.4k files. Reviewing the first page of results shows many repositories that haven't been updated in multiple years, so it is unclear which part of the `proc_macro_hygiene` feature they actually use (other parts of the feature were previously stabilized). ## Implementation ### Major parts > Summarize the major parts of the implementation and provide links into the code and to relevant PRs. The restriction is currently implemented as an explicit visitor of the proc macro input, rejecting outlined modules. The visitor is removed in this PR. ### Coverage > Summarize the test coverage of this feature. See the test changes attached to this PR. The added tests explicitly verify that proc macros receive `mod foo;` as an input without its body, to ensure that the behavior doesn't unintentionally change in the future. ### Outstanding bugs > What outstanding bugs involve this feature? List them. Should any block the stabilization? Discuss why or why not. None. ### Outstanding FIXMEs > What FIXMEs are still in the code for that feature and why is it OK to leave them there? None. ### Tool changes > What changes must be made to our other tools to support this feature. Has this work been done? Link to any relevant PRs and issues. Generally none. The feature is a minor addition to the inputs that (proc) macros accept, and should not require special handling in any of the tools. rust-analyzer has some existing limitations around handling modules transformed by proc macros (e.g. (on stable) auto-complete does not work if a proc macro transforms `mod foo {}` to `mod foo;`). It works fine for simple case where the the proc macro leaves the `mod foo;` mostly unmodified. ### Breaking changes > If this stabilization represents a known breaking change, link to the crater report, the analysis of the crater report, and to all PRs we've made to ecosystem projects affected by this breakage. Discuss any limitations of what we're able to know about or to fix. N/A: The feature is not expected to cause any breakage. ## Type system, opsem N/A: This feature only affects the AST, not type checking. ## Common interactions ### Temporaries > Does this feature introduce new expressions that can produce temporaries? What are the scopes of those temporaries? No. ### Drop order > Does this feature raise questions about the order in which we should drop values? Talk about the decisions made here and how they're consistent with our earlier decisions. No. ### Pre-expansion / post-expansion > Does this feature raise questions about what should be accepted pre-expansion (e.g. in code covered by `#[cfg(false)]`) versus what should be accepted post-expansion? What decisions were made about this? No. (At least nothing that doesn't equally apply to existing stable proc macros). ### Edition hygiene > If this feature is gated on an edition, how do we decide, in the context of the edition hygiene of tokens, whether to accept or reject code. E.g., what token do we use to decide? N/A: Not gated on an edition. ### SemVer implications > Does this feature create any new ways in which library authors must take care to prevent breaking downstreams when making minor-version releases? Describe these. Are these new hazards "major" or "minor" according to [RFC 1105](https://rust-lang.github.io/rfcs/1105-api-evolution.html)? No. ### Exposing other features > Are there any other unstable features whose behavior may be exposed by this feature in any way? What features present the highest risk of that? No. ## History > List issues and PRs that are important for understanding how we got here. See the RFC History section above. ## Acknowledgments > Summarize contributors to the feature by name for recognition and so that those people are notified about the stabilization. Does anyone who worked on this _not_ think it should be stabilized right now? We'd like to hear about that if so. - @alexcrichton for the work towards the initial proc-macro expansion. - @petrochenkov for significant work on hygiene and expansion since then. ## Open items > List any known items that have not yet been completed and that should be before this is stabilized. None. --- r? @petrochenkov for an initial review (feel free to edit the stabilization report if you like)
Stabilize `#[my_macro] mod foo;` (part of `proc_macro_hygiene`) # Stabilization report ## Summary Allow `mod foo;` (an "outlined" module) in the input of attribute and derive (proc) macros. Tracking: - rust-lang#54727 - The `proc_macro_hygiene` feature covers multiple related features. This PR proposes only a partial stabilization Reference PRs: - rust-lang/reference#2284 ### What is stabilized > Describe each behavior being stabilized and give a short example of code that will now be accepted. Currently, `mod foo;` (an outlined / non-inline) module is explicitly rejected in the input of derive and attribute (proc) macros. This PR removes the check, allowing `mod foo;`. The module is provided to the macro as it is written in the source code, that is in its "not-yet-loaded" form without a body. ```rust // The following was previously rejected and will now compile. #[my_proc_macro] mod foo; #[my_proc_macro] fn bar() { #[path = "foo.rs"] mod foo; } #[derive(MyTrait)] struct Foo([u8; { #[path = "foo.rs"] mod foo; 0 }]); ``` ### What isn't stabilized > Describe any parts of the feature not being stabilized. Talk about what we might want to do later and what doors are being left open for that. If what we're not stabilizing might lead to surprises for users, talk about that in particular. The remaining parts of `proc_macro_hygiene` are not stabilized (specifically, attribute macros in places where they aren't currently allowed). ## Design ### Reference > What updates are needed to the Reference? Link to each PR. If the Reference is missing content needed for describing this feature, discuss that. rust-lang/reference#2284 ### RFC history > What RFCs have been accepted for this feature? This feature has not been explicitly specified by any RFC. - Ahead of the `proc_macro` stabilization, `#[my_macro] mod foo;` specifically was feature gated. - As a side effect of rust-lang#52319, the (unstable) behavior changed: - Before that PR, `mod foo;` was actually provided to macros as the loaded `mod foo { ... }` - After that PR, `mod foo;` was provided to macros as-is. - In 2019 (after the `proc_macro` stabilization, rust-lang#66078 made `mod foo` _**anywhere**_ in the input to derive or attribute proc macros unstable. - Before that PR, only proc macros directly applied to `mod foo;` were unstable. ### Answers to unresolved questions > What questions were left unresolved by the RFC? How have they been answered? Link to any relevant lang decisions. N/A ### Post-RFC changes > What other user-visible changes have occurred since the RFC was accepted? Describe both changes that the lang team accepted (and link to those decisions) as well as changes that are being presented to the team for the first time in this stabilization report. See the RFC history above. ### Key points > What decisions have been most difficult and what behaviors to be stabilized have proved most contentious? Summarize the major arguments on all sides and link to earlier documents and discussions. The main design decision is whether outlined modules should be passed to proc macros with or without their body. Arguments in favor of "without their body": - (To the best of my knowledge) the current compiler architecture makes "without their body" the natural choice. - To [quote](rust-lang#54727 (comment)) @petrochenkov, "it's clear that the current behavior of `#[my_macro] mod foo;` (the "not-yet-loaded" version of the module is passed to the macro) is natural to the current setup, stable and is not going to change". - At least I would be somewhat surprised to see a macro (which is operating on the AST) able to see or affect code outside the current file. I would assume by default that they get as input exactly what I have written. - Function-like procedural macros operate on arbitrary token trees. `mod foo;` in their input is not even recognized by the compiler and thus not expanded. Attribute and derive proc macros also receiving the unexpanded `mod foo;` in their input makes the two consistent. - Other syntactic constructs, e.g. other macros like `inlcude!("foo.rs")`, are also not expanded before being passed to proc macros. Providing modules without their body would be consistent with that. - Providing `mod foo;` to proc macros with its body would (as far as I know) be the only case where a proc macro would not get exactly what is written in the source code as its input. Arguments in favor of "with their body": - `#[my_macro] mod foo;` and `#[my_macro] mod foo {}` would have different behavior, which means that in the presence of proc macros, migrating an inline module to an outlined one would no longer be a purely syntactic change, but could affect the behavior of the proc macro. - `#[my_macro] mod foo;` and `#![my_macro]` in `foo.rs` ([currently unstable](rust-lang#54726)) would have different behavior. - Other attributes already have different behaviors depending on whether they are applied as inner vs outer attributes. For example `#[cfg(false)]` on a module as an outer attribute means the corresponding file can be missig or have syntax errors, while using it as an inner attribute requires the file to be syntactically valid. - Some macros may need access to (or change) the module body to function properly. ### Nightly extensions > Are there extensions to this feature that remain unstable? How do we know that we are not accidentally committing to those? No. ### Doors closed > What doors does this stabilization close for later changes to the language? E.g., does this stabilization make any other RFCs, lang experiments, or known in-flight proposals more difficult or impossible to do later? None. If, in the future, there is a desire to make the module body of outlined modules available to proc macros, there are multiple ways to add that in a backward-compatible way (e.g. an explicit opt-in during the proc macro registration, or an API in the `proc_macro` crate). ## Feedback ### Call for testing > Has a "call for testing" been done? If so, what feedback was received? No. ### Nightly use > Do any known nightly users use this feature? Counting instances of `#![feature(FEATURE_NAME)]` on GitHub with grep might be informative. Searching Github [produces](https://github.com/search?q=%22feature%28proc_macro_hygiene%29%22+language%3ARust+&type=code) 4.4k files. Reviewing the first page of results shows many repositories that haven't been updated in multiple years, so it is unclear which part of the `proc_macro_hygiene` feature they actually use (other parts of the feature were previously stabilized). ## Implementation ### Major parts > Summarize the major parts of the implementation and provide links into the code and to relevant PRs. The restriction is currently implemented as an explicit visitor of the proc macro input, rejecting outlined modules. The visitor is removed in this PR. ### Coverage > Summarize the test coverage of this feature. See the test changes attached to this PR. The added tests explicitly verify that proc macros receive `mod foo;` as an input without its body, to ensure that the behavior doesn't unintentionally change in the future. ### Outstanding bugs > What outstanding bugs involve this feature? List them. Should any block the stabilization? Discuss why or why not. None. ### Outstanding FIXMEs > What FIXMEs are still in the code for that feature and why is it OK to leave them there? None. ### Tool changes > What changes must be made to our other tools to support this feature. Has this work been done? Link to any relevant PRs and issues. Generally none. The feature is a minor addition to the inputs that (proc) macros accept, and should not require special handling in any of the tools. rust-analyzer has some existing limitations around handling modules transformed by proc macros (e.g. (on stable) auto-complete does not work if a proc macro transforms `mod foo {}` to `mod foo;`). It works fine for simple case where the the proc macro leaves the `mod foo;` mostly unmodified. ### Breaking changes > If this stabilization represents a known breaking change, link to the crater report, the analysis of the crater report, and to all PRs we've made to ecosystem projects affected by this breakage. Discuss any limitations of what we're able to know about or to fix. N/A: The feature is not expected to cause any breakage. ## Type system, opsem N/A: This feature only affects the AST, not type checking. ## Common interactions ### Temporaries > Does this feature introduce new expressions that can produce temporaries? What are the scopes of those temporaries? No. ### Drop order > Does this feature raise questions about the order in which we should drop values? Talk about the decisions made here and how they're consistent with our earlier decisions. No. ### Pre-expansion / post-expansion > Does this feature raise questions about what should be accepted pre-expansion (e.g. in code covered by `#[cfg(false)]`) versus what should be accepted post-expansion? What decisions were made about this? No. (At least nothing that doesn't equally apply to existing stable proc macros). ### Edition hygiene > If this feature is gated on an edition, how do we decide, in the context of the edition hygiene of tokens, whether to accept or reject code. E.g., what token do we use to decide? N/A: Not gated on an edition. ### SemVer implications > Does this feature create any new ways in which library authors must take care to prevent breaking downstreams when making minor-version releases? Describe these. Are these new hazards "major" or "minor" according to [RFC 1105](https://rust-lang.github.io/rfcs/1105-api-evolution.html)? No. ### Exposing other features > Are there any other unstable features whose behavior may be exposed by this feature in any way? What features present the highest risk of that? No. ## History > List issues and PRs that are important for understanding how we got here. See the RFC History section above. ## Acknowledgments > Summarize contributors to the feature by name for recognition and so that those people are notified about the stabilization. Does anyone who worked on this _not_ think it should be stabilized right now? We'd like to hear about that if so. - @alexcrichton for the work towards the initial proc-macro expansion. - @petrochenkov for significant work on hygiene and expansion since then. ## Open items > List any known items that have not yet been completed and that should be before this is stabilized. None. --- r? @petrochenkov for an initial review (feel free to edit the stabilization report if you like)
Stabilize `#[my_macro] mod foo;` (part of `proc_macro_hygiene`) # Stabilization report ## Summary Allow `mod foo;` (an "outlined" module) in the input of attribute and derive (proc) macros. Tracking: - rust-lang#54727 - The `proc_macro_hygiene` feature covers multiple related features. This PR proposes only a partial stabilization Reference PRs: - rust-lang/reference#2284 ### What is stabilized > Describe each behavior being stabilized and give a short example of code that will now be accepted. Currently, `mod foo;` (an outlined / non-inline) module is explicitly rejected in the input of derive and attribute (proc) macros. This PR removes the check, allowing `mod foo;`. The module is provided to the macro as it is written in the source code, that is in its "not-yet-loaded" form without a body. ```rust // The following was previously rejected and will now compile. #[my_proc_macro] mod foo; #[my_proc_macro] fn bar() { #[path = "foo.rs"] mod foo; } #[derive(MyTrait)] struct Foo([u8; { #[path = "foo.rs"] mod foo; 0 }]); ``` ### What isn't stabilized > Describe any parts of the feature not being stabilized. Talk about what we might want to do later and what doors are being left open for that. If what we're not stabilizing might lead to surprises for users, talk about that in particular. The remaining parts of `proc_macro_hygiene` are not stabilized (specifically, attribute macros in places where they aren't currently allowed). ## Design ### Reference > What updates are needed to the Reference? Link to each PR. If the Reference is missing content needed for describing this feature, discuss that. rust-lang/reference#2284 ### RFC history > What RFCs have been accepted for this feature? This feature has not been explicitly specified by any RFC. - Ahead of the `proc_macro` stabilization, `#[my_macro] mod foo;` specifically was feature gated. - As a side effect of rust-lang#52319, the (unstable) behavior changed: - Before that PR, `mod foo;` was actually provided to macros as the loaded `mod foo { ... }` - After that PR, `mod foo;` was provided to macros as-is. - In 2019 (after the `proc_macro` stabilization, rust-lang#66078 made `mod foo` _**anywhere**_ in the input to derive or attribute proc macros unstable. - Before that PR, only proc macros directly applied to `mod foo;` were unstable. ### Answers to unresolved questions > What questions were left unresolved by the RFC? How have they been answered? Link to any relevant lang decisions. N/A ### Post-RFC changes > What other user-visible changes have occurred since the RFC was accepted? Describe both changes that the lang team accepted (and link to those decisions) as well as changes that are being presented to the team for the first time in this stabilization report. See the RFC history above. ### Key points > What decisions have been most difficult and what behaviors to be stabilized have proved most contentious? Summarize the major arguments on all sides and link to earlier documents and discussions. The main design decision is whether outlined modules should be passed to proc macros with or without their body. Arguments in favor of "without their body": - (To the best of my knowledge) the current compiler architecture makes "without their body" the natural choice. - To [quote](rust-lang#54727 (comment)) @petrochenkov, "it's clear that the current behavior of `#[my_macro] mod foo;` (the "not-yet-loaded" version of the module is passed to the macro) is natural to the current setup, stable and is not going to change". - At least I would be somewhat surprised to see a macro (which is operating on the AST) able to see or affect code outside the current file. I would assume by default that they get as input exactly what I have written. - Function-like procedural macros operate on arbitrary token trees. `mod foo;` in their input is not even recognized by the compiler and thus not expanded. Attribute and derive proc macros also receiving the unexpanded `mod foo;` in their input makes the two consistent. - Other syntactic constructs, e.g. other macros like `inlcude!("foo.rs")`, are also not expanded before being passed to proc macros. Providing modules without their body would be consistent with that. - Providing `mod foo;` to proc macros with its body would (as far as I know) be the only case where a proc macro would not get exactly what is written in the source code as its input. Arguments in favor of "with their body": - `#[my_macro] mod foo;` and `#[my_macro] mod foo {}` would have different behavior, which means that in the presence of proc macros, migrating an inline module to an outlined one would no longer be a purely syntactic change, but could affect the behavior of the proc macro. - `#[my_macro] mod foo;` and `#![my_macro]` in `foo.rs` ([currently unstable](rust-lang#54726)) would have different behavior. - Other attributes already have different behaviors depending on whether they are applied as inner vs outer attributes. For example `#[cfg(false)]` on a module as an outer attribute means the corresponding file can be missig or have syntax errors, while using it as an inner attribute requires the file to be syntactically valid. - Some macros may need access to (or change) the module body to function properly. ### Nightly extensions > Are there extensions to this feature that remain unstable? How do we know that we are not accidentally committing to those? No. ### Doors closed > What doors does this stabilization close for later changes to the language? E.g., does this stabilization make any other RFCs, lang experiments, or known in-flight proposals more difficult or impossible to do later? None. If, in the future, there is a desire to make the module body of outlined modules available to proc macros, there are multiple ways to add that in a backward-compatible way (e.g. an explicit opt-in during the proc macro registration, or an API in the `proc_macro` crate). ## Feedback ### Call for testing > Has a "call for testing" been done? If so, what feedback was received? No. ### Nightly use > Do any known nightly users use this feature? Counting instances of `#![feature(FEATURE_NAME)]` on GitHub with grep might be informative. Searching Github [produces](https://github.com/search?q=%22feature%28proc_macro_hygiene%29%22+language%3ARust+&type=code) 4.4k files. Reviewing the first page of results shows many repositories that haven't been updated in multiple years, so it is unclear which part of the `proc_macro_hygiene` feature they actually use (other parts of the feature were previously stabilized). ## Implementation ### Major parts > Summarize the major parts of the implementation and provide links into the code and to relevant PRs. The restriction is currently implemented as an explicit visitor of the proc macro input, rejecting outlined modules. The visitor is removed in this PR. ### Coverage > Summarize the test coverage of this feature. See the test changes attached to this PR. The added tests explicitly verify that proc macros receive `mod foo;` as an input without its body, to ensure that the behavior doesn't unintentionally change in the future. ### Outstanding bugs > What outstanding bugs involve this feature? List them. Should any block the stabilization? Discuss why or why not. None. ### Outstanding FIXMEs > What FIXMEs are still in the code for that feature and why is it OK to leave them there? None. ### Tool changes > What changes must be made to our other tools to support this feature. Has this work been done? Link to any relevant PRs and issues. Generally none. The feature is a minor addition to the inputs that (proc) macros accept, and should not require special handling in any of the tools. rust-analyzer has some existing limitations around handling modules transformed by proc macros (e.g. (on stable) auto-complete does not work if a proc macro transforms `mod foo {}` to `mod foo;`). It works fine for simple case where the the proc macro leaves the `mod foo;` mostly unmodified. ### Breaking changes > If this stabilization represents a known breaking change, link to the crater report, the analysis of the crater report, and to all PRs we've made to ecosystem projects affected by this breakage. Discuss any limitations of what we're able to know about or to fix. N/A: The feature is not expected to cause any breakage. ## Type system, opsem N/A: This feature only affects the AST, not type checking. ## Common interactions ### Temporaries > Does this feature introduce new expressions that can produce temporaries? What are the scopes of those temporaries? No. ### Drop order > Does this feature raise questions about the order in which we should drop values? Talk about the decisions made here and how they're consistent with our earlier decisions. No. ### Pre-expansion / post-expansion > Does this feature raise questions about what should be accepted pre-expansion (e.g. in code covered by `#[cfg(false)]`) versus what should be accepted post-expansion? What decisions were made about this? No. (At least nothing that doesn't equally apply to existing stable proc macros). ### Edition hygiene > If this feature is gated on an edition, how do we decide, in the context of the edition hygiene of tokens, whether to accept or reject code. E.g., what token do we use to decide? N/A: Not gated on an edition. ### SemVer implications > Does this feature create any new ways in which library authors must take care to prevent breaking downstreams when making minor-version releases? Describe these. Are these new hazards "major" or "minor" according to [RFC 1105](https://rust-lang.github.io/rfcs/1105-api-evolution.html)? No. ### Exposing other features > Are there any other unstable features whose behavior may be exposed by this feature in any way? What features present the highest risk of that? No. ## History > List issues and PRs that are important for understanding how we got here. See the RFC History section above. ## Acknowledgments > Summarize contributors to the feature by name for recognition and so that those people are notified about the stabilization. Does anyone who worked on this _not_ think it should be stabilized right now? We'd like to hear about that if so. - @alexcrichton for the work towards the initial proc-macro expansion. - @petrochenkov for significant work on hygiene and expansion since then. ## Open items > List any known items that have not yet been completed and that should be before this is stabilized. None. --- r? @petrochenkov for an initial review (feel free to edit the stabilization report if you like)
Stabilize `#[my_macro] mod foo;` (part of `proc_macro_hygiene`) # Stabilization report ## Summary Allow `mod foo;` (an "outlined" module) in the input of attribute and derive (proc) macros. Tracking: - rust-lang#54727 - The `proc_macro_hygiene` feature covers multiple related features. This PR proposes only a partial stabilization Reference PRs: - rust-lang/reference#2284 ### What is stabilized > Describe each behavior being stabilized and give a short example of code that will now be accepted. Currently, `mod foo;` (an outlined / non-inline) module is explicitly rejected in the input of derive and attribute (proc) macros. This PR removes the check, allowing `mod foo;`. The module is provided to the macro as it is written in the source code, that is in its "not-yet-loaded" form without a body. ```rust // The following was previously rejected and will now compile. #[my_proc_macro] mod foo; #[my_proc_macro] fn bar() { #[path = "foo.rs"] mod foo; } #[derive(MyTrait)] struct Foo([u8; { #[path = "foo.rs"] mod foo; 0 }]); ``` ### What isn't stabilized > Describe any parts of the feature not being stabilized. Talk about what we might want to do later and what doors are being left open for that. If what we're not stabilizing might lead to surprises for users, talk about that in particular. The remaining parts of `proc_macro_hygiene` are not stabilized (specifically, attribute macros in places where they aren't currently allowed). ## Design ### Reference > What updates are needed to the Reference? Link to each PR. If the Reference is missing content needed for describing this feature, discuss that. rust-lang/reference#2284 ### RFC history > What RFCs have been accepted for this feature? This feature has not been explicitly specified by any RFC. - Ahead of the `proc_macro` stabilization, `#[my_macro] mod foo;` specifically was feature gated. - As a side effect of rust-lang#52319, the (unstable) behavior changed: - Before that PR, `mod foo;` was actually provided to macros as the loaded `mod foo { ... }` - After that PR, `mod foo;` was provided to macros as-is. - In 2019 (after the `proc_macro` stabilization, rust-lang#66078 made `mod foo` _**anywhere**_ in the input to derive or attribute proc macros unstable. - Before that PR, only proc macros directly applied to `mod foo;` were unstable. ### Answers to unresolved questions > What questions were left unresolved by the RFC? How have they been answered? Link to any relevant lang decisions. N/A ### Post-RFC changes > What other user-visible changes have occurred since the RFC was accepted? Describe both changes that the lang team accepted (and link to those decisions) as well as changes that are being presented to the team for the first time in this stabilization report. See the RFC history above. ### Key points > What decisions have been most difficult and what behaviors to be stabilized have proved most contentious? Summarize the major arguments on all sides and link to earlier documents and discussions. The main design decision is whether outlined modules should be passed to proc macros with or without their body. Arguments in favor of "without their body": - (To the best of my knowledge) the current compiler architecture makes "without their body" the natural choice. - To [quote](rust-lang#54727 (comment)) @petrochenkov, "it's clear that the current behavior of `#[my_macro] mod foo;` (the "not-yet-loaded" version of the module is passed to the macro) is natural to the current setup, stable and is not going to change". - At least I would be somewhat surprised to see a macro (which is operating on the AST) able to see or affect code outside the current file. I would assume by default that they get as input exactly what I have written. - Function-like procedural macros operate on arbitrary token trees. `mod foo;` in their input is not even recognized by the compiler and thus not expanded. Attribute and derive proc macros also receiving the unexpanded `mod foo;` in their input makes the two consistent. - Other syntactic constructs, e.g. other macros like `inlcude!("foo.rs")`, are also not expanded before being passed to proc macros. Providing modules without their body would be consistent with that. - Providing `mod foo;` to proc macros with its body would (as far as I know) be the only case where a proc macro would not get exactly what is written in the source code as its input. Arguments in favor of "with their body": - `#[my_macro] mod foo;` and `#[my_macro] mod foo {}` would have different behavior, which means that in the presence of proc macros, migrating an inline module to an outlined one would no longer be a purely syntactic change, but could affect the behavior of the proc macro. - `#[my_macro] mod foo;` and `#![my_macro]` in `foo.rs` ([currently unstable](rust-lang#54726)) would have different behavior. - Other attributes already have different behaviors depending on whether they are applied as inner vs outer attributes. For example `#[cfg(false)]` on a module as an outer attribute means the corresponding file can be missig or have syntax errors, while using it as an inner attribute requires the file to be syntactically valid. - Some macros may need access to (or change) the module body to function properly. ### Nightly extensions > Are there extensions to this feature that remain unstable? How do we know that we are not accidentally committing to those? No. ### Doors closed > What doors does this stabilization close for later changes to the language? E.g., does this stabilization make any other RFCs, lang experiments, or known in-flight proposals more difficult or impossible to do later? None. If, in the future, there is a desire to make the module body of outlined modules available to proc macros, there are multiple ways to add that in a backward-compatible way (e.g. an explicit opt-in during the proc macro registration, or an API in the `proc_macro` crate). ## Feedback ### Call for testing > Has a "call for testing" been done? If so, what feedback was received? No. ### Nightly use > Do any known nightly users use this feature? Counting instances of `#![feature(FEATURE_NAME)]` on GitHub with grep might be informative. Searching Github [produces](https://github.com/search?q=%22feature%28proc_macro_hygiene%29%22+language%3ARust+&type=code) 4.4k files. Reviewing the first page of results shows many repositories that haven't been updated in multiple years, so it is unclear which part of the `proc_macro_hygiene` feature they actually use (other parts of the feature were previously stabilized). ## Implementation ### Major parts > Summarize the major parts of the implementation and provide links into the code and to relevant PRs. The restriction is currently implemented as an explicit visitor of the proc macro input, rejecting outlined modules. The visitor is removed in this PR. ### Coverage > Summarize the test coverage of this feature. See the test changes attached to this PR. The added tests explicitly verify that proc macros receive `mod foo;` as an input without its body, to ensure that the behavior doesn't unintentionally change in the future. ### Outstanding bugs > What outstanding bugs involve this feature? List them. Should any block the stabilization? Discuss why or why not. None. ### Outstanding FIXMEs > What FIXMEs are still in the code for that feature and why is it OK to leave them there? None. ### Tool changes > What changes must be made to our other tools to support this feature. Has this work been done? Link to any relevant PRs and issues. Generally none. The feature is a minor addition to the inputs that (proc) macros accept, and should not require special handling in any of the tools. rust-analyzer has some existing limitations around handling modules transformed by proc macros (e.g. (on stable) auto-complete does not work if a proc macro transforms `mod foo {}` to `mod foo;`). It works fine for simple case where the the proc macro leaves the `mod foo;` mostly unmodified. ### Breaking changes > If this stabilization represents a known breaking change, link to the crater report, the analysis of the crater report, and to all PRs we've made to ecosystem projects affected by this breakage. Discuss any limitations of what we're able to know about or to fix. N/A: The feature is not expected to cause any breakage. ## Type system, opsem N/A: This feature only affects the AST, not type checking. ## Common interactions ### Temporaries > Does this feature introduce new expressions that can produce temporaries? What are the scopes of those temporaries? No. ### Drop order > Does this feature raise questions about the order in which we should drop values? Talk about the decisions made here and how they're consistent with our earlier decisions. No. ### Pre-expansion / post-expansion > Does this feature raise questions about what should be accepted pre-expansion (e.g. in code covered by `#[cfg(false)]`) versus what should be accepted post-expansion? What decisions were made about this? No. (At least nothing that doesn't equally apply to existing stable proc macros). ### Edition hygiene > If this feature is gated on an edition, how do we decide, in the context of the edition hygiene of tokens, whether to accept or reject code. E.g., what token do we use to decide? N/A: Not gated on an edition. ### SemVer implications > Does this feature create any new ways in which library authors must take care to prevent breaking downstreams when making minor-version releases? Describe these. Are these new hazards "major" or "minor" according to [RFC 1105](https://rust-lang.github.io/rfcs/1105-api-evolution.html)? No. ### Exposing other features > Are there any other unstable features whose behavior may be exposed by this feature in any way? What features present the highest risk of that? No. ## History > List issues and PRs that are important for understanding how we got here. See the RFC History section above. ## Acknowledgments > Summarize contributors to the feature by name for recognition and so that those people are notified about the stabilization. Does anyone who worked on this _not_ think it should be stabilized right now? We'd like to hear about that if so. - @alexcrichton for the work towards the initial proc-macro expansion. - @petrochenkov for significant work on hygiene and expansion since then. ## Open items > List any known items that have not yet been completed and that should be before this is stabilized. None. --- r? @petrochenkov for an initial review (feel free to edit the stabilization report if you like)
To track whether module declarations are inline I added a field
inline: booltoast::Mod. The main use case is for pretty to know whether it should render the items associated with the module, but perhaps there are use cases for this information to not be forgotten in the AST.