This came up during our review of ILM. Many participants were surprised by the reordering of the options when they were de/selected:

As a user, if I misclick I want to be able to click again to correct my misclick, but the reordering of the options makes this an exercise in frustration, because it just results in another misclick. If I've scrolled down in the list, then it looks like the option I clicked disappeared, which is also confusing.

GitHub's UI has a nice UX:

What's particularly nice is that the selected items are moved to the top after the user has finished making their selection and closed the list: