A newly added test from #32493 shows that some models fail their generate() call comparison when inputs are input_ids or inputs_embeds. Mamba-2 which was added in #32080 seems to fail this test.