The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Yes, good point, I thought about that too. I opted to keep both because the new method is much more readable and intuitive, so it doesn't really add too much noise in my opinion. It also documents why we are using the long approach, and provides a hint (via the link to the github issue) to remove that case when we have full support of isin() for mps at some point in the future. But sure, we can keep just one if that's preferred :)

I'm pro the if/else: it serves as a reminder to revisit the code in the future, when mps supports this more readable code path

I'm not sure we should be doing this - the lack of support is really a torch issue, and not a transformers one. There's many operations which are (unfortunately) unsupported by the mps backend, and I wouldn't want to see a tonne of if/else statements across the library to support this. At the moment, PYTORCH_ENABLE_MPS_FALLBACK=1 is the standard approach to try and use mps, which I believe resolves this issue

I don't really mind this as it should be mostly for Llama3, but only if that produces the correct outputs. If not, then we should just recommend to use mlx?
@pcuenca does this produce gibberish or same as on cuda?

Thanks so much for the quick fix. Could this get a release?

Yes, will do this asap, just waiting to make sure we don't have other failures.