Skip to content

[Models] Llama2 fix#333

Merged
Aalanli merged 12 commits intohidet-org:mainfrom
Aalanli:llama2-fix
Jul 31, 2023
Merged

[Models] Llama2 fix#333
Aalanli merged 12 commits intohidet-org:mainfrom
Aalanli:llama2-fix

Conversation

@Aalanli
Copy link
Copy Markdown
Contributor

@Aalanli Aalanli commented Jul 27, 2023

Fix llama2 and add test for num_heads != num_key_value_heads.

@Aalanli
Copy link
Copy Markdown
Contributor Author

Aalanli commented Jul 28, 2023

This is quite strange, it seems that while the tests I added works on my local machine, it does not work on the CI server. I am investigating if the transformer versions are the same or not.

@yaoyaoding
Copy link
Copy Markdown
Member

Have added a CI step to print out the version of installed python packages at #334 .

@Aalanli
Copy link
Copy Markdown
Contributor Author

Aalanli commented Jul 29, 2023

Right, it seems that the transformer version is 4.29.2, when the latest one is 4.31.0.

@yaoyaoding
Copy link
Copy Markdown
Member

I see. The github ci will cache the python package (so that we do not need to reinstall the large packages like pytorch for every run), and this also makes some packages not up to date.

Just cleared the python cache manually on the ci server.

@Aalanli
Copy link
Copy Markdown
Contributor Author

Aalanli commented Jul 29, 2023

It appears that updating transformers broke torch dynamo for bert, specifically for dynamic=True. This has nothing to do with our backend, though.

@yaoyaoding
Copy link
Copy Markdown
Member

I see, could you skip the corresponding tests in this PR? Let's wait for the fix from pytorch people.

AL added 2 commits July 30, 2023 10:48
@Aalanli Aalanli merged commit 0532154 into hidet-org:main Jul 31, 2023
@Aalanli Aalanli deleted the llama2-fix branch July 31, 2023 00:07
soodoshll pushed a commit to soodoshll/hidet that referenced this pull request Jul 31, 2023
Fix llama2 and add test for num_heads != num_key_value_heads.

---------

Co-authored-by: Allan Lin <allan.lin@centml.ai>
soodoshll added a commit to soodoshll/hidet that referenced this pull request Aug 4, 2023
author su <soodoshll@gmail.com> 1690484279 -0400
committer Qidong Su <soodoshll@gmail.com> 1691188805 +0000

parent a73fc3e
author su <soodoshll@gmail.com> 1690484279 -0400
committer Qidong Su <soodoshll@gmail.com> 1691184810 +0000

shard spec data structure

[CI] List package versions in ci (hidet-org#334)

update

update

update

update

update

update

update

[Models] Llama2 fix (hidet-org#333)

Fix llama2 and add test for num_heads != num_key_value_heads.

---------

Co-authored-by: Allan Lin <allan.lin@centml.ai>

update

update

update

update

upate

update

update

update

update

fix

update

update

update

update'

remove llama

update

dependency

update

update

update

update

update

update

add llama

update

update

format

refactor

update

update

shard spec data structure

update

update

update

update

update

update

update

format

refactor

fix

update

fix

fix
soodoshll added a commit to soodoshll/hidet that referenced this pull request Aug 15, 2023
author su <soodoshll@gmail.com> 1690484279 -0400
committer Qidong Su <soodoshll@gmail.com> 1691188805 +0000

parent a73fc3e
author su <soodoshll@gmail.com> 1690484279 -0400
committer Qidong Su <soodoshll@gmail.com> 1691184810 +0000

shard spec data structure

[CI] List package versions in ci (hidet-org#334)

update

update

update

update

update

update

update

[Models] Llama2 fix (hidet-org#333)

Fix llama2 and add test for num_heads != num_key_value_heads.

---------

Co-authored-by: Allan Lin <allan.lin@centml.ai>

update

update

update

update

upate

update

update

update

update

fix

update

update

update

update'

remove llama

update

dependency

update

update

update

update

update

update

add llama

update

update

format

refactor

update

update

shard spec data structure

update

update

update

update

update

update

update

format

refactor

fix

update

fix

fix
vadiklyutiy pushed a commit that referenced this pull request Jul 22, 2024
vadiklyutiy pushed a commit that referenced this pull request Jul 23, 2024
vadiklyutiy pushed a commit that referenced this pull request Dec 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants