🐛 Describe the bug

Hi, it appears that using do_constant_folding=True in the ONNX export will undo some weight deduplication. For example, an nn.Linear weight will go from

&

to

effectively transposing the weight. Given that the DeduplicateInitializersByDataPtr relies on the tensor size, in case the shared weight has a different size (e.g. an embedding weight), the deduplication pass will fail.

It seems to me that the initializer deduplication should happen before the constant folding, and constant folding should be done only for non-shared weights.

WDTY @justinchuby @BowenBao ?

Thank you!

Repro:

pip install optimum
optimum-cli export onnx -m bigscience/bloom-560m bloom_onnx --no-post-process

and inspect the output with netron

Versions

Both on 2.0.1 and nightly