Skip to content

Improve locale comparison#98743

Merged
Repiteo merged 1 commit intogodotengine:masterfrom
dbnicholson:improve-compare-locales
Nov 12, 2024
Merged

Improve locale comparison#98743
Repiteo merged 1 commit intogodotengine:masterfrom
dbnicholson:improve-compare-locales

Conversation

@dbnicholson
Copy link
Copy Markdown
Contributor

Since 9456454, country specific locales are always preferred over country-less locales even when the OS locale country doesn't match. For example, running the Godot editor with locale es_ES will result in the es_AR locale being chosen even though the es locale would be better.

The change happened because the score of the es_AR locale and the es locale are the same when comparing to es_ES. Change this by parsing locale strings into a Locale structure and decreasing the score when script, country or variant are set in both but not matched. For the es_ES case, this causes the es_AR score to be decreased since the countries don't match. On the other hand, the es locale is not decreased since it doesn't specify a country.

Fixes: #90677

@dbnicholson dbnicholson requested review from a team as code owners November 1, 2024 23:53
@AThousandShips AThousandShips added this to the 4.x milestone Nov 2, 2024
@dbnicholson dbnicholson force-pushed the improve-compare-locales branch from 22495ae to 4a272e8 Compare November 3, 2024 20:57
@timothyqiu
Copy link
Copy Markdown
Member

When matching fr_CH from ["fr", "fr_CA"], fr is correctly picked.

When matching zh_HK from ["zh", "zh_CN"], both these two options score 3, which is not expected.

@dbnicholson
Copy link
Copy Markdown
Contributor Author

When matching zh_HK from ["zh", "zh_CN"], both these two options score 3, which is not expected.

Wow, that is very unusual. It's because of the this part that sets the script and/or country when comparing locales:

  • zh_HK -> zh_Hant_HK
  • zh -> zh_Hans_CN
  • zh_CN -> zh_Hans_CN

In other words, zh is equivalent to zh_CN and the score gets deducted by 2 because both the script and country mismatch. I'll add test cases with comments for that.

@dbnicholson dbnicholson force-pushed the improve-compare-locales branch from 4a272e8 to 954508f Compare November 7, 2024 13:18
@dbnicholson
Copy link
Copy Markdown
Contributor Author

I added test cases for zh-HK vs zh-CN and zh-CN vs zh. I also fleshed out the test comments a bit more for why the scores are the way they are. It would be nice if you could call standardize_locale with the defaults added outside of TranslationServer, but that's a separate issue.

Since 9456454, country specific locales are always preferred over
country-less locales even when the OS locale country doesn't match. For
example, running the Godot editor with locale es_ES will result in the
es_AR locale being chosen even though the es locale would be better.

The change happened because the score of the es_AR locale and the es
locale are the same when comparing to es_ES. Change this by parsing
locale strings into a Locale structure and decreasing the score when
script, country or variant are set in both but not matched. For the
es_ES case, this causes the es_AR score to be decreased since the
countries don't match. On the other hand, the es locale is not decreased
since it doesn't specify a country.
@dbnicholson dbnicholson force-pushed the improve-compare-locales branch from 954508f to 2dd409f Compare November 12, 2024 13:06
@akien-mga akien-mga modified the milestones: 4.x, 4.4 Nov 12, 2024
@dbnicholson
Copy link
Copy Markdown
Contributor Author

It would be nice if you could call standardize_locale with the defaults added outside of TranslationServer, but that's a separate issue.

#98972 addresses that (and will likely cause a conflict with this if merged).

@Repiteo Repiteo merged commit 27b394c into godotengine:master Nov 12, 2024
@dbnicholson dbnicholson deleted the improve-compare-locales branch November 12, 2024 15:36
@Repiteo
Copy link
Copy Markdown
Contributor

Repiteo commented Nov 12, 2024

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Translations GetText - Fallback from language with country code to language without a country code not working

6 participants