Skip to content

Incomplete syntax handling for proc-macros produces invalid TokenStream with {/} chars as a Punct rather than a Group #18244

@Veetaha

Description

@Veetaha

rust-analyzer version: rust-analyzer version: 0.3.2129-standalon

rustc version: rustc 1.81.0 (eeb90cda1 2024-09-04)

editor or extension: VSCode v0.3.2129

I've stumbled on a bug when using my proc-macro #[bon::builder].

The minimal reproduction of this is:

#[my_lovely_macro]
fn example() {
    if 1
}

where my_lovely_macro is defined as this

use proc_macro::{TokenStream, TokenTree};

#[proc_macro_attribute]
pub fn my_lovely_macro(_params: TokenStream, item: TokenStream) -> TokenStream {
    let tokens: Vec<_> = item.clone().into_iter().collect();
    let TokenTree::Group(fn_block) = &tokens[3] else {
        unreachable!();
    };
    let fn_block_tokens: Vec<_> = fn_block.stream().into_iter().collect();

    if fn_block_tokens.len() > 2 {
        // This is the case where RA inserted a dummy {} block
        panic!("{fn_block_tokens:#?}");
    }

    item
}

If you take a look at the panic message from RA hints

it looks like this:

proc-macro panicked: [
    Ident {
        ident: "if",
        span: SpanData { range: 38..40, anchor: SpanAnchor(FileId(16777216), 2), ctx: SyntaxContextId(0) },
    },
    Literal {
        kind: Integer,
        symbol: "1",
        suffix: None,
        span: SpanData { range: 41..42, anchor: SpanAnchor(FileId(16777216), 2), ctx: SyntaxContextId(0) },
    },
    Punct {
        ch: '{',
        spacing: Alone,
        span: SpanData { range: 0..0, anchor: SpanAnchor(FileId(16777216), 4294967294), ctx: SyntaxContextId(0) },
    },
    Punct {
        ch: '}',
        spacing: Alone,
        span: SpanData { range: 0..0, anchor: SpanAnchor(FileId(16777216), 4294967294), ctx: SyntaxContextId(0) },
    },
]

Notice how there are two Punct items for { and } braces in this list? This is an invalid token tree. syn fails to parse this syntax, because curly braces are never meant to be Punct characters, they must always be balanced and they are only expected as part of a Group token tree.

This produced a panic in my bon::builder macro visible to the user writing the code if they have an incomplete if somewhere while writing the function body (the entire function is underlined in red and no IDE hints are provided). I'll fix that panic in bon separately to make my macro more bug-resilient to situations like this, but RA should also fix this.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions