Skip to content

[deps] update compel to fix black output with weight=0; also use new downweighting algorithm#2961

Merged
blessedcoolant merged 2 commits intoinvoke-ai:mainfrom
damian0815:fixfeat_black_weight0_new_downweighting
Mar 16, 2023
Merged

[deps] update compel to fix black output with weight=0; also use new downweighting algorithm#2961
blessedcoolant merged 2 commits intoinvoke-ai:mainfrom
damian0815:fixfeat_black_weight0_new_downweighting

Conversation

@damian0815
Copy link
Copy Markdown
Contributor

@damian0815 damian0815 commented Mar 15, 2023

Update compel to 1.0.0.

This fixes #2832.

It also changes the way downweighting is applied. In particular, downweighting should now be much better and more controllable.

From the compel changelog:

Downweighting now works by applying an attention mask to remove the downweighted tokens, rather than literally removing them from the sequence. This behaviour is the default, but the old behaviour can be re-enabled by passing downweight_mode=DownweightMode.REMOVE on init of the Compel instance.

Formerly, downweighting a token worked by both multiplying the weighting of the token's embedding, and doing an inverse-weighted blend with a copy of the token sequence that had the downweighted tokens removed. The intuition is that as weight approaches zero, the tokens being downweighted should be actually removed from the sequence. However, removing the tokens resulted in the positioning of all downstream tokens becoming messed up. The blend ended up blending a lot more than just the tokens in question.

As of v1.0.0, taking advice from @keturn and @bonlime (damian0815/compel#7) the procedure is by default different. Downweighting still involves a blend but what is blended is a version of the token sequence with the downweighted tokens masked out, rather than removed. This correctly preserves positioning embeddings of the other tokens.

@blessedcoolant blessedcoolant merged commit c341dca into invoke-ai:main Mar 16, 2023
@damian0815 damian0815 changed the title update compel to fix black screens and use new downweighting algorithm [deps] update compel to fix black output with weight=0; also use new downweighting algorithm Mar 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[bug]: Prompt with term weighted 0 results in an all-black image

2 participants