Skip to content
This repository was archived by the owner on Dec 14, 2023. It is now read-only.

Convert diffusers to original MS format#52

Merged
ExponentialML merged 17 commits intoExponentialML:mainfrom
kabachuha:convert-from-diffusers
Apr 17, 2023
Merged

Convert diffusers to original MS format#52
ExponentialML merged 17 commits intoExponentialML:mainfrom
kabachuha:convert-from-diffusers

Conversation

@kabachuha
Copy link
Copy Markdown
Contributor

@kabachuha kabachuha commented Apr 17, 2023 •
edited
Loading

  • Convert the In/Out blocks
  • Convert the Text encoder
  • Convert Unet ResBlocks
  • Testing

closes #29

@kabachuha
Copy link
Copy Markdown
Contributor Author

kabachuha commented Apr 17, 2023 •
edited
Loading

ModelStructure reference

Diffusers:

message.txt

ModelScope:

model_info.txt

@kabachuha kabachuha changed the title [WIP] Convert diffusers to original MS format Convert diffusers to original MS format Apr 17, 2023
@kabachuha kabachuha marked this pull request as ready for review April 17, 2023 12:57
@kabachuha
Copy link
Copy Markdown
Contributor Author

kabachuha commented Apr 17, 2023

The conversion works!

python convert_diffusers_to_original_ms_text_to_video.py --model_path animov --checkpoint_path text2video_pytorch_model.pth --clip_checkpoint_path open_clip_pytorch_model.bin 
Initializing the conversion map
Converting the UNET
Saving UNET
Saving CLIP
Operation successfull

The only thing left is to load the model and see if it works

@kabachuha
Copy link
Copy Markdown
Contributor Author

kabachuha commented Apr 17, 2023

Fixed all the keys mismatch

@kabachuha
Copy link
Copy Markdown
Contributor Author

kabachuha commented Apr 17, 2023

All complete

alUntitled.mp4

@ExponentialML ExponentialML merged commit 9eac110 into ExponentialML:main Apr 17, 2023
@ExponentialML
Copy link
Copy Markdown
Owner

ExponentialML commented Apr 17, 2023

This was on my todo list, but you knocked it out of the park. Thank you very much!

@kabachuha kabachuha deleted the convert-from-diffusers branch April 17, 2023 22:23
@kabachuha kabachuha restored the convert-from-diffusers branch April 22, 2023 11:52
@sanatorios
Copy link
Copy Markdown

sanatorios commented May 21, 2023

This is amazing but I'm not really sure about the needed process. Would love to convert strangeman3107's animov-512x fine-tuned model into MS format as well. How it could be possiblle?

@howardgriffin
Copy link
Copy Markdown

howardgriffin commented Aug 2, 2023 •
edited
Loading

Hi, I have noticed that the newest version(v3.0.0) of 'convert_diffusers_to_original_ms_text_to_video.py' doesn't save clip_checkpoint.
What's more, this script doesn't save the configuration.json as https://github.com/kabachuha/sd-webui-text2video mentioned.
So，I cann't test my own trained model with sd_webui. How to solve my problem? @kabachuha @ExponentialML

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Using a trained model on webui

4 participants

@kabachuha @ExponentialML @sanatorios @howardgriffin