Add basic support for localization#4729
Merged
bk2204 merged 9 commits intogit-lfs:mainfrom Nov 15, 2021
Merged
Conversation
6a7e486 to
7b4a06d
Compare
We'd like to support localization in Git LFS. To do so, add the gotext package, which implement gettext-compatible localization.
We'd like to support localizing Git LFS. We have the gotext package,
which supports several different ways of getting GNU gettext-compatible
locale strings. However, we need some central message catalog we can
use.
Let's create a singleton by the name of Tr. That means that we can
translate a string by writing something like this:
fmt.Println(tr.Tr.Get("This is a message about %s.", subject))
These strings can be automatically extracted by the xgotext tool
available from the package
github.com/leonelquinteros/gotext/cli/xgotext. While currently quite
slow, it does understand Go syntax, unlike the standard GNU gettext
suite.
We initialize it based on the locale. Since Go does not use the
standard C library, we have to emulate its behavior, as outlined in
locale(7). If our locale is "en_DK.UTF-8", we first look for "en_DK"
and then "en", in that order. This preserves a fallback for languages
which are similar across most speakers, like English and Spanish, and
can share a translation, but for locales which cannot, such pt_PT and
pt_BR, or zh_TW and zh_CN, this ensures we prefer a more appropriate
locale.
Introduce a variable called "locales" which will be used to store
Base64-encoded MO files (compiled Gettext files) for each locale. We
can still load external locales from the standard directory
(/usr/share/locale), but this preserves the behavior of providing a
single, relocatable binary, which people like. If this becomes a
problem in the future, we can revisit it. Population of this variable
will come in a future commit.
Currently, Git LFS has the benefit of being a single, relocatable binary. While it is customary to store locales in /usr/share/locales on Unix systems, this isn't necessarily the case on Windows, and we'd like to preserve the ability of users to move the binary where they like. As a result, let's add a program which can be run by go generate that embeds these translations into the binary. They are Base64-encoded to make handling them a little easier and avoiding need to write giant strings of backslashed escape sequences. The reader will note that this is based off the related command to embed the manual pages.
As the first thing we do when we run a command, let's initialize the locale. This will let us print any sort of error or warning messages in the user's preferred language if we have any problems.
chrisd8088
approved these changes
Nov 15, 2021
Member
chrisd8088
left a comment
There was a problem hiding this comment.
This is great stuff, thank you!
|
|
||
| //go:generate go run ../tr/trgen/trgen.go | ||
|
|
||
| var Tr = gotext.NewLocale("/usr/share/locale", "en") |
Member
There was a problem hiding this comment.
Just out of interest, how will this operate on Windows?
Member
Author
There was a problem hiding this comment.
It will not, but since we don't expect external locale files at all for Git LFS, it shouldn't be a problem. It may be that a Linux distro chooses to ship these externally, but that wouldn't work on Windows anyway since Windows doesn't have a canonical place for them, so again, the fact that this does nothing shouldn't be a problem.
Let's make sure we build the translations into the binary when possible. Specifically, if the msgfmt command exists, let's compile the .po files into .mo files and then embed their contents into the binary using the generator command to do that. Let's also ignore the built .mo files and the generated .pot template files so we don't check them in.
It will take some time for us to get the codebase translated. However, as a test of our codepaths and as a demonstration of what to do, let's introduce a single translation, plus a simple Spanish translation that translates only that string. This allows an invocation like the following to print a simple translated message: LC_ALL=es_ES.UTF-8 git lfs filter-process
When building the default image in CI, or a release build, ensure that the gettext package is installed so we can test locales. We don't include it in all CI runs to ensure that we also test building without localization. Note that on Windows, these tools should always be installed as part of the Git for Windows environment, since they are needed for Git, and we would have no way to install them anyway.
chrisd8088
added a commit
to chrisd8088/git-lfs
that referenced
this pull request
Dec 10, 2024
Since commit 6285716 in PR git-lfs#4729 we have installed the gettext library and utilities in our CI and release GitHub Actions jobs, on both macOS and Ubuntu Linux platforms, while on Windows we rely on the library and utilities being installed in advance as part of the Git for Windows environment. At present, though, the macOS runners provided by GitHub Actions are always provisioned with a pre-installed version of a recent release of the gettext library, as packaged by Homebrew, which includes the msgfmt(1) program used by our Makefile, so we no longer need to install the "gettext" Homebrew formula ourselves.
chrisd8088
added a commit
to chrisd8088/git-lfs
that referenced
this pull request
Dec 15, 2024
Since commit 6285716 in PR git-lfs#4729 we have installed the gettext library and utilities in our CI and release GitHub Actions jobs, on both macOS and Ubuntu Linux platforms, while on Windows we rely on the library and utilities being installed in advance as part of the Git for Windows environment. At present, though, the macOS runners provided by GitHub Actions are always provisioned with a pre-installed version of a recent release of the gettext library, as packaged by Homebrew, which includes the msgfmt(1) program used by our Makefile, so we no longer need to install the "gettext" Homebrew formula ourselves.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Most people on the planet do not speak English. We'd like Git LFS to be accessible to them as well, so add basic support for localization. Each commit explains its rationale, but this introduces a localization framework and but a single localized string. However, this allows us to continue localizing more and more of Git LFS and permits translators to start when they're comfortable.