Skip to content

fix: FakeTensors appearing in get_attr calls#2669

Merged
gs-olive merged 2 commits intopytorch:mainfrom
gs-olive:draft_fixes_2658
Mar 20, 2024
Merged

fix: FakeTensors appearing in get_attr calls#2669
gs-olive merged 2 commits intopytorch:mainfrom
gs-olive:draft_fixes_2658

Conversation

@gs-olive
Copy link
Copy Markdown
Contributor

@gs-olive gs-olive commented Mar 5, 2024

Description

  • Register all constants as model parameters, which do not get fake-ified by the active FakeTensor context
  • Buffers and other constant registrations can be fake-ified, which is problematic for TRT tracing

Fixes #2658

Type of change

Please delete options that are not relevant and/or add your own.

  • Bug fix (non-breaking change which fixes an issue)

Checklist:

  • [ x ] My code follows the style guidelines of this project (You can use the linters)
  • [ x ] I have performed a self-review of my own code
  • [ x ] I have commented my code, particularly in hard-to-understand areas and hacks
  • [ x ] I have made corresponding changes to the documentation
  • [ - ] I have added tests to verify my fix or my feature
    • Pending CI verification on existing models
  • [ x ] New and existing unit tests pass locally with my changes
  • [ x ] I have added the relevant labels to my PR in so that relevant reviewers are notified

@gs-olive gs-olive self-assigned this Mar 5, 2024
@github-actions github-actions bot added component: lowering Issues re: The lowering / preprocessing passes component: conversion Issues re: Conversion stage component: api [Python] Issues re: Python API component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths labels Mar 5, 2024
@gs-olive
fix: FakeTensors appearing in get_attr calls
88f7895 
- Register all constants as model parameters, which do not get
fake-ified by the active FakeTensor context
- Buffers and other constant registrations can be fake-ified, which is
problematic for TRT tracing
zewenli98
zewenli98 approved these changes Mar 6, 2024
View reviewed changes
Copy link
Copy Markdown
Collaborator

@zewenli98 zewenli98 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! I'm just wondering if PyTorch has documentation to illustrate what kind of types will be fake-ified?

@gs-olive
Copy link
Copy Markdown
Contributor Author

gs-olive commented Mar 8, 2024

Thanks for the review - I haven't found any concrete documentation on this topic; this is mainly a workaround that I've found to work for a few key models and not break others. This documentation on FakeTensor is what I am using currently.

@bowang007 bowang007 self-requested a review March 18, 2024 22:59
bowang007
bowang007 approved these changes Mar 18, 2024
View reviewed changes
Copy link
Copy Markdown
Collaborator

@bowang007 bowang007 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@bowang007
Copy link
Copy Markdown
Collaborator

bowang007 commented Mar 18, 2024

@gs-olive can you merge this one?
This can make us a lot easier every time using the nightly container.
Thanks!

This was referenced Mar 18, 2024
Merged
Merged
@gs-olive
Copy link
Copy Markdown
Contributor Author

gs-olive commented Mar 20, 2024

Since testing is not working in CI, running the Dynamo test suite locally to verify

peri044
peri044 approved these changes Mar 20, 2024
View reviewed changes
Copy link
Copy Markdown
Collaborator

@peri044 peri044 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@gs-olive gs-olive merged commit 766c270 into pytorch:main Mar 20, 2024
@gs-olive gs-olive deleted the draft_fixes_2658 branch March 20, 2024 01:10
peri044 pushed a commit that referenced this pull request Apr 19, 2024
@gs-olive @peri044
fix: FakeTensors appearing in get_attr calls (#2669)
815751b
laikhtewari pushed a commit that referenced this pull request May 24, 2024
@gs-olive @laikhtewari
fix: FakeTensors appearing in get_attr calls (#2669)
9cbdda3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@peri044 peri044 peri044 approved these changes

@zewenli98 zewenli98 zewenli98 approved these changes

@bowang007 bowang007 bowang007 approved these changes

@narendasan narendasan Awaiting requested review from narendasan

Assignees

@gs-olive gs-olive

Labels

cla signed component: api [Python] Issues re: Python API component: conversion Issues re: Conversion stage component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths component: lowering Issues re: The lowering / preprocessing passes

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

🐛 [Bug] Issue in conversion when parameters/buffers are moved during compilation

5 participants

@gs-olive @bowang007 @peri044 @zewenli98 @facebook-github-bot