Steps to reproduce the issue

See #23222 which I now close as this one will replace it.
It is also a ping...

Set your browser preferred language to en-GB
Install a clean Joomla 4.
Select en-US as language.
Process installation until getting to the remove view.
Pass steps one after the other. en-US is kept as language and direction until getting to the index.php?view=remove&layout=default page where you can choose to Customise Installation or Complete and Open Site, or Complete & Open Admin
Look at source: the language has changed to en-GB.

I have also done the test by adding a 3.9.6 fr-FR language in installation.
In that case I have set the browser language to fr-FR and also used indifferently en-GB or en-US as installation language.
In that case, when getting to the remove view, the language is switched to fr-FR.

The conclusion is that the session language is lost and code checks again for LanguageHelper::detectLanguage(); when getting to the "remove" view.

I think it is time to correct that bug. Could not find how.