Skip to content

feat: Add support for is_causal argument in attention#2780

Merged
gs-olive merged 3 commits intopytorch:mainfrom
gs-olive:is_causal_support
Apr 30, 2024
Merged

feat: Add support for is_causal argument in attention#2780
gs-olive merged 3 commits intopytorch:mainfrom
gs-olive:is_causal_support

Conversation

@gs-olive
Copy link
Copy Markdown
Contributor

@gs-olive gs-olive commented Apr 26, 2024

Description

  • Add support for is_causal argument in attention converter

Type of change

  • New feature (non-breaking change which adds functionality)

Checklist:

  • [ x ] My code follows the style guidelines of this project (You can use the linters)
  • [ x ] I have performed a self-review of my own code
  • [ x ] I have commented my code, particularly in hard-to-understand areas and hacks
  • [ x ] I have made corresponding changes to the documentation
  • [ x ] I have added tests to verify my fix or my feature
  • [ x ] New and existing unit tests pass locally with my changes
  • [ x ] I have added the relevant labels to my PR in so that relevant reviewers are notified

@gs-olive gs-olive requested a review from peri044 April 26, 2024 00:29
@gs-olive gs-olive self-assigned this Apr 26, 2024
@github-actions github-actions bot added component: tests Issues re: Tests component: lowering Issues re: The lowering / preprocessing passes component: conversion Issues re: Conversion stage component: converters Issues re: Specific op converters component: api [Python] Issues re: Python API component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths labels Apr 26, 2024
@gs-olive gs-olive force-pushed the is_causal_support branch from 62878a3 to a81b7f7 Compare April 26, 2024 18:56
peri044
peri044 reviewed Apr 30, 2024
View reviewed changes
py/torch_tensorrt/dynamo/conversion/impl/attention.py
scale,
)

if is_causal:
Copy link
Copy Markdown
Collaborator

@peri044 peri044 Apr 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't support dynamic shapes right ?

Copy link
Copy Markdown
Contributor Author

@gs-olive gs-olive Apr 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current implementation does not, no

@peri044
Copy link
Copy Markdown
Collaborator

peri044 commented Apr 30, 2024 •
edited
Loading

Changes LGTM

@gs-olive gs-olive merged commit e6f9aa2 into pytorch:main Apr 30, 2024
@gs-olive gs-olive deleted the is_causal_support branch April 30, 2024 22:34
laikhtewari pushed a commit that referenced this pull request May 24, 2024
@gs-olive @laikhtewari
feat: Add support for is_causal argument in attention (#2780)
588ec96
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@peri044 peri044 peri044 left review comments

Assignees

@gs-olive gs-olive

Labels

cla signed component: api [Python] Issues re: Python API component: conversion Issues re: Conversion stage component: converters Issues re: Specific op converters component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths component: lowering Issues re: The lowering / preprocessing passes component: tests Issues re: Tests

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@gs-olive @peri044 @facebook-github-bot