Skip to content

"AssertionError: Guard check failed" on PyTorch nightlies after 2025-01-22 when running torchao tests #145733

@vkuzo

Description

@vkuzo

🐛 Describe the bug

Running the following command introduces an assertion error on recent versions of PyTorch nightlies:

// run a test in torchao repo
// test source code: https://github.com/pytorch/ao/blob/main/test/prototype/test_smoothquant.py

> pytest test/prototype/test_smoothquant.py -s -x

...
        if not output_graph.export:
            if not self.guard_manager.check(output_graph.local_scope):
                reasons = get_guard_fail_reason_helper(
                    self.guard_manager,  # type: ignore[arg-type]
                    output_graph.local_scope,
                    CompileContext.current_compile_id(),
                )
>               raise AssertionError(f"Guard check failed: {reasons}")
E               AssertionError: Guard check failed: 0/0: ___check_metadata_139942440334048_c0/0                      
E               
E               
E               You can suppress this exception and fall back to eager by setting:
E                   import torch._dynamo
E                   torch._dynamo.config.suppress_errors = True

Full output: https://gist.github.com/vkuzo/b50ac59aa0936fd072250d7544aaa2a9

The nightly from 20250122 does not have this failure, but the nightlies from 20250123 and after do.

Versions

https://gist.github.com/vkuzo/ddcaa43948a96cf63e58fb71fbdcbf68

cc @ezyang @gchanan @zou3519 @kadeng @msaroufim @chauhang @penguinwu @voznesenskym @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @amjames

Metadata

Metadata

Assignees

No one assigned

    Labels

    high prioritymodule: dynamooncall: pt2triagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate module

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions