As far as I understand, this file cannot be directly created in the container because by the moment the container has started, the configuration will have been read, and the newly created file will be ignored. Right?

Yes, this is done according to documentation and is a standard way to provide custom configs to dockerized services:

Using a custom MySQL configuration file
...
This will start a new container some-mysql where the MySQL instance uses the combined startup settings from /etc/mysql/my.cnf and /etc/mysql/conf.d/config-file.cnf, with settings from the latter taking precedence.

Maybe we can try altering the user instead if you don't like linking a config: https://dev.mysql.com/doc/refman/8.0/en/upgrading-from-previous-series.html#upgrade-caching-sha2-password

No, it's fine leaving it as is. I didn't realize it's the right way to do things and started asking, then realized but decided to post the question to make sure.