Skip to content

Mark almost all strings for translation#4781

Merged
bk2204 merged 47 commits intogit-lfs:mainfrom
bk2204:translations
Jan 18, 2022
Merged

Mark almost all strings for translation#4781
bk2204 merged 47 commits intogit-lfs:mainfrom
bk2204:translations

Conversation

@bk2204
Copy link
Member

@bk2204 bk2204 commented Dec 14, 2021

This PR marks almost all of our strings for localization. In addition, it provides documentation to help contributors of all sorts more gracefully deal with translations and translated strings.

Finally, there is a fake locale which can be created for testing, i-reverse, to allow testing whether strings are localized or not. It consists of the English text, but each word (as in \w+) reversed. This is not built in by default because it would bloat the binary for little benefit.

Most of this series is merely boring invocations of adding tr.Tr.Get (or sometimes, tr.Tr.GetN). Occasionally, there are more interesting conversions, such as the need to move variables defined at file scope into functions to be translated, since the locale object will not have been properly initialized until after startup (and the file-scope strings would be run through the locale object before main is run).

In most places, we turn a fmt.Errorf(...) into an errors.New(tr.Tr.Get(...)) so that we don't evaluate the translated string a second time.

There is one particular matter which is of interest. The Get function is not a complete analogue of fmt.Sprintf. Unlike fmt.Sprintf, Get does not process formatting patterns unless there are arguments. That is, Get("%s") returns %s, and Get("%s", "foo") returns foo. However, the linter built into Go will complain if we do Get("%s"), since it thinks an argument will be needed, and this causes make test to fail.

Our solution is that in a couple places in the code when we need a format string to be localized, we write something like f := Get("%%s"), and then later on do fmt.Sprintf(f, "arg"). This looks bizarre, and it is, and I've filed leonelquinteros/gotext#57 about it, so hopefully we can upgrade and then fix this in the future. However, for now, we'll just go with it.

Due to the amount of commits in this series, boring as most of them might be, I fully expect that this won't receive a review until 2022, and that's fine. There's no rush.

@bk2204 bk2204 force-pushed the translations branch 3 times, most recently from de39416 to 0ae6bbc Compare December 14, 2021 20:57
@bk2204 bk2204 marked this pull request as ready for review December 15, 2021 17:54
@bk2204 bk2204 requested a review from a team as a code owner December 15, 2021 17:54
Copy link
Member

@chrisd8088 chrisd8088 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a huge effort, thank you so much!

I think my suggestions and concerns fall into a couple of areas:

  • I believe there may be some strings in the commands/* source files outside of the commands/command_*.go files, such as commands/uploader.go and commands/lockverifier.go. These should probably also be converted to the new translation format.
  • I'm uncertain if it's correct that the xgotext output file, po/default.pot, uses backticks to delimit multi-line strings. If by chance that's a problem in xgotext, we should probably submit a patch for that upstream, and then we wouldn't need as much parsing logic in the Ruby helper script.
  • Because this PR highlights all our strings, I noticed that there's a some heterogeneity to the prefixes we use, particular. Some error strings start with fatal: but most do not; while warnings may start with warning:, Warning:, or WARNING: (and there's also ERROR: in some places, plus 'Abort:). We should probably take this opportunity to make these all consistent so we aren't giving potential translators extra challenges. And I wondered if the few places where we output a bare Usage:` string we should change to output actual text.
  • The commands/command_*.go files have many help-text strings for command-line options, which are passed to the Cobra command-line flag functions like cmd.Flags().StringVarP(). But I have a suspicion that none of these are actually available to users, and if that proves to be true, I think maybe we could treat them more like internal comments and save translators the trouble of tackling all of them.

I think those were the main things I noticed, but there are at least two where I'm a bit uncertain and would appreciate a second opinion!

Thank you again so much for kicking off these big projects and getting them to a workable state!

@bk2204
Copy link
Member Author

bk2204 commented Jan 3, 2022

Okay, except where specified above, I've made all the changes you suggested in a series of fixup commits, plus a few new commits for the files I've missed. When you're happy with things, I can squash them down.

Copy link
Member

@chrisd8088 chrisd8088 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to go through and mark as many conversations resolved as possible, and I've added a few additional notes, but not many!

The main thing I noticed on a second read-through is that we're a bit wobbly in how we capitalize, or don't, the first word of many messages. I'm not sure what to say about that and if it's worth tackling here or not. It obviously doesn't matter to the functionality of the client; it's more a question, perhaps, of whether we want to provide a reliable guide to translators as to whether they should follow suit or not.

@bk2204
Copy link
Member Author

bk2204 commented Jan 14, 2022

I believe I've addressed the last few things. I think we should perhaps do a follow-up to make the casing consistent here instead of doing it in this PR, since this is already quite large.

Copy link
Member

@chrisd8088 chrisd8088 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is definitely looking good! I went through and added a last bunch of notes (some of which you might want to just ignore), but I didn't want to hold this up asking for another re-review round.

bk2204 and others added 18 commits January 18, 2022 17:03
We'd like to make it easier for folks to write code that is easy to
localize and to have localizations that are simple and intuitive.  Let's
add some documentation on how to write good localized strings, how to
perform the extraction of strings, and general advice both to
contributors of code and translations.
Turn several separate strings into one long string since they are
essentially a single message and having a single string is easier to
translate.
Turn several separate strings into one long string since they are
essentially a single message and having a single string is easier to
translate.
Turn several strings with "(s)" on them into proper calls to GetN.
Turn several strings with "(s)" on them into proper calls to GetN.
Translate the messages of our fsck failures, but not the prefix or kind,
which are designed for automatic parsing.
Move the string for lockRemoteHelp variable into the actual help output.
We cannot translate a global variable because at initialization time the
locale string won't have been loaded and we won't be able to load the
proper string.
Move the string for lockRemoteHelp variable into the actual help output.
We cannot translate a global variable because at initialization time the
locale string won't have been loaded and we won't be able to load the
proper string.
Move the string for lockRemoteHelp variable into the actual help output.
We cannot translate a global variable because at initialization time the
locale string won't have been loaded and we won't be able to load the
proper string.  Since the variable is now unused, remove it.
Co-authored-by: Chris Darroch <chrisd8088@github.com>
Consolidate multiple strings into one to make translating easier,
especially since the translator will want to align the text so that it
lines up.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Jan 27, 2022
In commit 08774bf of
PR git-lfs#4781 two message strings in the tq/api_test.go file
were made translatable; however, since this only contains
Go tests, we can reverse that change.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Jan 28, 2022
In commit 08774bf of
PR git-lfs#4781 two message strings in the tq/api_test.go file
were made translatable; however, since this only contains
Go tests, we can reverse that change.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Jan 28, 2022
In commit 1ddc2f2 of
PR git-lfs#4781 text translation support was added for the
"git lfs ext" command and a multi-line message was
concatenated.  However, the latter part of this text should
be fixed for all users, since it displays a listing of
the extension's settings values using defined terms.

We therefore separate the first line of the message and
let that be translated, as it contains the language-specific
header text.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Jan 28, 2022
In commit ff0f378 of
PR git-lfs#4781 a usage message for the "git lfs untrack" command was
made translatable; however, since this only contains a fixed
command example which references the man pages, we can avoid
translating it.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Jan 28, 2022
In commit 1ddc2f2 of
PR git-lfs#4781 text translation support was added for the
"git lfs ext" command and a multi-line message was
concatenated.  However, the latter part of this text should
be fixed for all users, since it displays a listing of
the extension's settings values using defined terms.

We therefore separate the first line of the message and
let that be translated, as it contains the language-specific
header text.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Jan 28, 2022
In commit ff0f378 of
PR git-lfs#4781 a usage message for the "git lfs untrack" command was
made translatable; however, since this only contains a fixed
command example which references the man pages, we can avoid
translating it.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Jan 28, 2022
In commit 08774bf of
PR git-lfs#4781 two message strings in the tq/api_test.go file
were made translatable; however, since this only contains
Go tests, we can reverse that change.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Jan 28, 2022
In commit 1ddc2f2 of
PR git-lfs#4781 text translation support was added for the
"git lfs ext" command and a multi-line message was
concatenated.  However, the latter part of this text should
be fixed for all users, since it displays a listing of
the extension's settings values using defined terms.

We therefore separate the first line of the message and
let that be translated, as it contains the language-specific
header text.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Jan 28, 2022
In commit ff0f378 of
PR git-lfs#4781 a usage message for the "git lfs untrack" command was
made translatable; however, since this only contains a fixed
command example which references the man pages, we can avoid
translating it.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Jan 28, 2022
In commit c744476 of
PR git-lfs#4781 first part of a multi-line message was made
translatable, while the second part, which consists only of
an example "git config" command, was not.

That change is reasonable; however, some indentation
whitespace preceding the example command was lost, so we
restore it here.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Jan 28, 2022
In commit 08774bf of
PR git-lfs#4781 two message strings in the tq/api_test.go file
were made translatable; however, since this only contains
Go tests, we can reverse that change.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Jan 28, 2022
Following on from the changes in PR git-lfs#4781, we can make
additional message strings translatable using the
tr.Tr.Get() method.

Because this method performs printf(3)-style format string
parsing and interpolation, we can simplify some of the
surrounding calls, e.g., from fmt.Errorf() to errors.New(),
and from fmt.Fprintf() to fmt.Fprintln().  This ensures
that if either the translated text or any interpolated
arguments happen to contain character sequences that would
be interpreted as Go format specifiers (e.g., "%s" or "%d"),
these will not result in warnings such as "%!s(MISSING)"
in the output text.

Note also that we try to remove newlines from the message
strings were possible and change the surrounding calls to
append them instead, e.g., with fmt.Fprintln().
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Jan 30, 2022
In commit 1ddc2f2 of
PR git-lfs#4781 text translation support was added for the
"git lfs ext" command and a multi-line message was
concatenated.  However, the latter part of this text should
be fixed for all users, since it displays a listing of
the extension's settings values using defined terms.

We therefore separate the first line of the message and
let that be translated, as it contains the language-specific
header text.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Jan 30, 2022
In commit ff0f378 of
PR git-lfs#4781 a usage message for the "git lfs untrack" command was
made translatable; however, since this only contains a fixed
command example which references the man pages, we can avoid
translating it.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Jan 30, 2022
In commit c744476 of
PR git-lfs#4781 first part of a multi-line message was made
translatable, while the second part, which consists only of
an example "git config" command, was not.

That change is reasonable; however, some indentation
whitespace preceding the example command was lost, so we
restore it here.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Jan 30, 2022
In commit 08774bf of
PR git-lfs#4781 two message strings in the tq/api_test.go file
were made translatable; however, since this only contains
Go tests, we can reverse that change.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Jan 30, 2022
Following on from the changes in PR git-lfs#4781, we can make
additional message strings translatable using the
tr.Tr.Get() method.

Because this method performs printf(3)-style format string
parsing and interpolation, we can simplify some of the
surrounding calls, e.g., from fmt.Errorf() to errors.New(),
and from fmt.Fprintf() to fmt.Fprintln().  This ensures
that if either the translated text or any interpolated
arguments happen to contain character sequences that would
be interpreted as Go format specifiers (e.g., "%s" or "%d"),
these will not result in warnings such as "%!s(MISSING)"
in the output text.

Note also that we try to remove newlines from the message
strings were possible and change the surrounding calls to
append them instead, e.g., with fmt.Fprintln().
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Jan 30, 2022
In commit 1ddc2f2 of
PR git-lfs#4781 text translation support was added for the
"git lfs ext" command and a multi-line message was
concatenated.  However, the latter part of this text should
be fixed for all users, since it displays a listing of
the extension's settings values using defined terms.

We therefore separate the first line of the message and
let that be translated, as it contains the language-specific
header text.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Jan 30, 2022
In commit ff0f378 of
PR git-lfs#4781 a usage message for the "git lfs untrack" command was
made translatable; however, since this only contains a fixed
command example which references the man pages, we can avoid
translating it.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Jan 30, 2022
In commit c744476 of
PR git-lfs#4781 first part of a multi-line message was made
translatable, while the second part, which consists only of
an example "git config" command, was not.

That change is reasonable; however, some indentation
whitespace preceding the example command was lost, so we
restore it here.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Jan 30, 2022
In commit 08774bf of
PR git-lfs#4781 two message strings in the tq/api_test.go file
were made translatable; however, since this only contains
Go tests, we can reverse that change.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Jan 30, 2022
Following on from the changes in PR git-lfs#4781, we can make
additional message strings translatable using the
tr.Tr.Get() method.

Because this method performs printf(3)-style format string
parsing and interpolation, we can simplify some of the
surrounding calls, e.g., from fmt.Errorf() to errors.New(),
and from fmt.Fprintf() to fmt.Fprintln().  This ensures
that if either the translated text or any interpolated
arguments happen to contain character sequences that would
be interpreted as Go format specifiers (e.g., "%s" or "%d"),
these will not result in warnings such as "%!s(MISSING)"
in the output text.

Note also that we try to remove newlines from the message
strings were possible and change the surrounding calls to
append them instead, e.g., with fmt.Fprintln().
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Feb 28, 2025
In PR git-lfs#4781 we updated a number of packages to replace global string
variables with local variables so they could be translated via the
"tr" package's localization functions.

One concern raised in that PR pertained to the changes to the
lfshttp/errors.go source file, where a map of error format strings
is constructed in the defaultError() function, with each string
mapped to a specific HTTP status code.  These strings should contain
at least one "%s" verb, into which the HTTP request URL should be
interpolated.

In commit cda9c3c of PR git-lfs#4781 we moved
the map's initialization into the defaultError() function of our
"lfshttp" package, and added a call to the Get() method of the Tr
structure from our "tr" package around each string's definition.
However, we deferred the interpolation of the request URL to a
subsequent call to a string formatting function, with the intent that
we would first select the correct localized format string from the
map based on the HTTP request's status code, and then interpolate
the request's URL into the final error message.

Further, if a request's status code does not correspond to a string
in the map, our defaultError() function creates a generic error
message into which we try to interpolate both the request's URL and
its status code.  For these generic messages, the intent of the
changes in commit cda9c3c was that
we first interpolate the status code, and then treat the resultant
string as if it had been found in the defaultErrors map, meaning it
should still contain one "%s" verb into which the request URL can be
interpolated.

Such a design implies that in the generic error messages we need to
use a "%d" verb for the status code, and a doubly-escaped "%%s" verb
for the request URL, since we want the latter to be converted to a
regular "%s" verb when the string is first used as a format string.

This design exposed an unexpected issue, which is described in
leonelquinteros/gotext#57.  In trying to work around this issue, though,
we introduced a bug of our own whereby the generic messages are passed
through three formatting functions rather than two, and in doing so
the "%%s" verbs are eliminated before we attempt to interpolate the
HTTP request's URL, leading to further errors.

The single commit in PR git-lfs#5822 attempts to fix this problem, but without
complete success.  We now adopt the changes from that PR's commit and
resolve the remaining problems so we should always supply an appropriate
set of arguments for the number of formatting verbs in both types of
messages in the defaultError() function.

The confusion here stems from the issue reported in
leonelquinteros/gotext#57, namely that the Printf() function of the
"github.com/leonelquinteros/gotext" package behaves differently
depending on whether or not it is passed any arguments other than
a format string.  When no arguments are passed, the format string
is not parsed and any verbs are left unaltered, whereas when
at least one additional argument is passed, the format string is
parsed and the trailing arguments interpolated into it.

The Tr global variable in our "tr" package is a Locale structure
from the "gotext" package.  Starting with commit
cda9c3c, we separately invoke this
variable's Get() method on each of the error messages defined in
the defaultErrors map, but without any other arguments.  Because
the Get() method does not parse the format string when no other
arguments are passed, in theory we ought to be able to define these
messages with regular "%s" verbs.  If we do so, though, the Go linter
complains that it appears we are passing format strings to the Get()
method without matching arguments for their "%s" verbs.

To work around this linter limitation, we currently use "%%s" verbs in
the error messages in the defaultErrors map, which are then passed
through the Get() method unchanged.  We then make a subsequent call
to the Sprintf() function from the Go standard library's "fmt" package,
passing the error message as the format string but without any other
arguments.  This call effectively just converts any "%%s" verbs to
"%s" verbs.

By contrast, the generic error messages in the defaultError() function
are passed to the Tr variable's Get() method with another argument,
the HTTP request's status code.  Due to the presence of the extra
argument, the Get() method parses the format string, replacing the "%d"
verb with the status code and converting the "%%s" verb into a "%s"
verb.  (Note that originally, in PR git-lfs#4781, we used the Sprintf()
function of the standard "fmt" package rather than the Get() method
of our Tr variable.  We corrected this oversight in commit
04abbd8 of PR git-lfs#4846, but while that
change allows the strings to be localized, it has no effect on the
handling of the formatting verbs.)

As with the error messages from the defaultErrors map, the generic
error messages are then also passed without any other arguments to the
Sprintf() function from the standard "fmt" package.  However, this call
does not have the same effect in both cases.  When the error message is
specific to an HTTP status code and has been selected from the
defaultErrors map, this call just converts the "%%s" verb into a "%s"
one, as noted above.  When the error message is a generic one, though,
that conversion has already been performed by the Tr structure's Get()
method.  The consequence is that the Sprintf() function replaces the
"%s" verb with "%!s(MISSING)", as described in the "fmt" package's
documentation:

  https://pkg.go.dev/fmt#hdr-Format_errors

In the final step of the defaultError() function we then pass the
result from this Sprintf() call to the Errof() function of our
"errors" package, and here we include the HTTP request's URL as a second
argument.  In the case where the error message has been selected from
the defaultErrors map, this last formatting function replaces the "%s"
verb with the URL.  In the case where the error message is a generic one,
though, the leading portion of the format error marker "%!s(MISSING)"
cannot be parsed as a verb, and so the "%!" is replaced with a full
reflection of all the internal elements of the URL structure into
string form.

Since these format error markers and reflection fields are not desirable
in our HTTP request error messages, we revise the defaultError() function
so it each type of error message is passed through only a single
formatting function, specifically the Tr variable's Get() method.
This allows us to use regular "%s" and "%d" formatting verbs exclusively
in all the messages.

Rather than call the Get() method on each string as it is defined in
the defaultErrors map, we only call the method once a specific message
is selected based on the HTTP request's status code, and we pass the
request's URL to the Get() method as another argument.  For the generic
error messages, we simply pass both the status code and the URL as
arguments to the Get() function.  Because we always pass either one or
two arguments to that function in addition to a format string, it always
performs the appropriate interpolation for us, and we avoid the need
to work around its behaviour when it is only passed a format string
without other arguments, as well as avoiding the use of any further
formatting functions like Sprintf().

Co-authored-by: Philip Peterson <pc.peterso@gmail.com>
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Feb 28, 2025
When we report an error, we often call the Errorf() function in our
custom "errors" package, which accepts both a format string and a
variable number of additional arguments.  The intent of this function
is for the additional arguments to be interpolated into the format
string, and then an error created using the string which results from
the interpolation.

In many instances we originally made use of the interpolation capability
of the Errorf() function by passing at least one additional argument to
it other than the format string.  In other cases, though, we used the
Errorf() function with just a single fixed string argument, so the
the New() function of the same "errors" package would have sufficed
instead.

In PR git-lfs#4781 we then updated our code to allow for localization of most
of our output messages by inserting calls to the Get() method of the Tr
structure of our "tr" package.  In the instances where we call the
Errorf() function, the consequence is that we now pass the format string
and any additional arguments to the Get() method, and then just pass
the string it returns to the Errorf() function as a single argument.

As of Go 1.24, if the Go version number in the "go.mod" file is set to
1.24 or higher, the "go vet" command now reports misuses of non-constant
strings as format strings.  This includes all the cases where we pass
the output from the Get() method of the Tr structure to an Errorf()
function as a single argument.  To resolve this concern, we revise
all such instances to use the New() function of our "errors" package
in place of the Errorf() function.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Feb 28, 2025
When we report an error, we often call the Wrapf() function in our
custom "errors" package, which accepts an existing error, a format
string, and a variable number of additional arguments.  The intent of
this function is for the additional arguments to be interpolated into
the format string, and then a new error created using the string which
results from the interpolation, combined with the initial error and
its message.

In many instances we originally made use of the interpolation capability
of the Wrapf() function by passing at least one additional argument to
it other than the existing error and the format string.  In other cases,
though, we used the Wrapf() function with just the existing error and
a fixed string argument, so the Wrap() function of the same "errors"
package would have sufficed instead.

In PR git-lfs#4781 updated our code to allow for localization of most of our
output messages by inserting calls to the Get() method of the Tr
structure of our "tr" package.  In the instances where we call the
Wrapf() function, the consequence is that we now pass the format string
and any additional arguments to the Get() method, and then just pass
the string it returns to the Wrapf() function as the format string
argument following the initial existing error argument.

As of Go 1.24, if the Go version number in the "go.mod" file is set to
1.24 or higher, the "go vet" command now reports misuses of non-constant
strings as format strings.  This includes all the cases where we pass
the output from the Get() method of the Tr structure to an Wrapf()
function as a format string without any additional arguments to be
interpolated.  To resolve this concern, we revise all such instances to
use the Wrap() function of our "errors" package in place of the Wrapf()
function.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Mar 5, 2025
In PR git-lfs#4781 we updated a number of packages to replace global string
variables with local variables so they could be translated via the
"tr" package's localization functions.

One concern raised in that PR pertained to the changes to the
lfshttp/errors.go source file, where a map of error format strings
is constructed in the defaultError() function, with each string
mapped to a specific HTTP status code.  These strings should contain
at least one "%s" verb, into which the HTTP request URL should be
interpolated.

In commit cda9c3c of PR git-lfs#4781 we moved
the map's initialization into the defaultError() function of our
"lfshttp" package, and added a call to the Get() method of the Tr
structure from our "tr" package around each string's definition.
However, we deferred the interpolation of the request URL to a
subsequent call to a string formatting function, with the intent that
we would first select the correct localized format string from the
map based on the HTTP request's status code, and then interpolate
the request's URL into the final error message.

Further, if a request's status code does not correspond to a string
in the map, our defaultError() function creates a generic error
message into which we try to interpolate both the request's URL and
its status code.  For these generic messages, the intent of the
changes in commit cda9c3c was that
we first interpolate the status code, and then treat the resultant
string as if it had been found in the defaultErrors map, meaning it
should still contain one "%s" verb into which the request URL can be
interpolated.

Such a design implies that in the generic error messages we need to
use a "%d" verb for the status code, and a doubly-escaped "%%s" verb
for the request URL, since we want the latter to be converted to a
regular "%s" verb when the string is first used as a format string.

This design exposed an unexpected issue, which is described in
leonelquinteros/gotext#57.  In trying to work around this issue, though,
we introduced a bug of our own whereby the generic messages are passed
through three formatting functions rather than two, and in doing so
the "%%s" verbs are eliminated before we attempt to interpolate the
HTTP request's URL, leading to further errors.

The single commit in PR git-lfs#5822 attempts to fix this problem, but without
complete success.  We now adopt the changes from that PR's commit and
resolve the remaining problems so we should always supply an appropriate
set of arguments for the number of formatting verbs in both types of
messages in the defaultError() function.

The confusion here stems from the issue reported in
leonelquinteros/gotext#57, namely that the Printf() function of the
"github.com/leonelquinteros/gotext" package behaves differently
depending on whether or not it is passed any arguments other than
a format string.  When no arguments are passed, the format string
is not parsed and any verbs are left unaltered, whereas when
at least one additional argument is passed, the format string is
parsed and the trailing arguments interpolated into it.

The Tr global variable in our "tr" package is a Locale structure
from the "gotext" package.  Starting with commit
cda9c3c, we separately invoke this
variable's Get() method on each of the error messages defined in
the defaultErrors map, but without any other arguments.  Because
the Get() method does not parse the format string when no other
arguments are passed, in theory we ought to be able to define these
messages with regular "%s" verbs.  If we do so, though, the Go linter
complains that it appears we are passing format strings to the Get()
method without matching arguments for their "%s" verbs.

To work around this linter limitation, we currently use "%%s" verbs in
the error messages in the defaultErrors map, which are then passed
through the Get() method unchanged.  We then make a subsequent call
to the Sprintf() function from the Go standard library's "fmt" package,
passing the error message as the format string but without any other
arguments.  This call effectively just converts any "%%s" verbs to
"%s" verbs.

By contrast, the generic error messages in the defaultError() function
are passed to the Tr variable's Get() method with another argument,
the HTTP request's status code.  Due to the presence of the extra
argument, the Get() method parses the format string, replacing the "%d"
verb with the status code and converting the "%%s" verb into a "%s"
verb.  (Note that originally, in PR git-lfs#4781, we used the Sprintf()
function of the standard "fmt" package rather than the Get() method
of our Tr variable.  We corrected this oversight in commit
04abbd8 of PR git-lfs#4846, but while that
change allows the strings to be localized, it has no effect on the
handling of the formatting verbs.)

As with the error messages from the defaultErrors map, the generic
error messages are then also passed without any other arguments to the
Sprintf() function from the standard "fmt" package.  However, this call
does not have the same effect in both cases.  When the error message is
specific to an HTTP status code and has been selected from the
defaultErrors map, this call just converts the "%%s" verb into a "%s"
one, as noted above.  When the error message is a generic one, though,
that conversion has already been performed by the Tr structure's Get()
method.  The consequence is that the Sprintf() function replaces the
"%s" verb with "%!s(MISSING)", as described in the "fmt" package's
documentation:

  https://pkg.go.dev/fmt#hdr-Format_errors

In the final step of the defaultError() function we then pass the
result from this Sprintf() call to the Errof() function of our
"errors" package, and here we include the HTTP request's URL as a second
argument.  In the case where the error message has been selected from
the defaultErrors map, this last formatting function replaces the "%s"
verb with the URL.  In the case where the error message is a generic one,
though, the leading portion of the format error marker "%!s(MISSING)"
cannot be parsed as a verb, and so the "%!" is replaced with a full
reflection of all the internal elements of the URL structure into
string form.

Since these format error markers and reflection fields are not desirable
in our HTTP request error messages, we revise the defaultError() function
so it each type of error message is passed through only a single
formatting function, specifically the Tr variable's Get() method.
This allows us to use regular "%s" and "%d" formatting verbs exclusively
in all the messages.

Rather than call the Get() method on each string as it is defined in
the defaultErrors map, we only call the method once a specific message
is selected based on the HTTP request's status code, and we pass the
request's URL to the Get() method as another argument.  For the generic
error messages, we simply pass both the status code and the URL as
arguments to the Get() function.  Because we always pass either one or
two arguments to that function in addition to a format string, it always
performs the appropriate interpolation for us, and we avoid the need
to work around its behaviour when it is only passed a format string
without other arguments, as well as avoiding the use of any further
formatting functions like Sprintf().

Co-authored-by: Philip Peterson <pc.peterso@gmail.com>
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Mar 5, 2025
When we report an error, we often call the Errorf() function in our
custom "errors" package, which accepts both a format string and a
variable number of additional arguments.  The intent of this function
is for the additional arguments to be interpolated into the format
string, and then an error created using the string which results from
the interpolation.

In many instances we originally made use of the interpolation capability
of the Errorf() function by passing at least one additional argument to
it other than the format string.  In other cases, though, we used the
Errorf() function with just a single fixed string argument, so the
the New() function of the same "errors" package would have sufficed
instead.

In PR git-lfs#4781 we then updated our code to allow for localization of most
of our output messages by inserting calls to the Get() method of the Tr
structure of our "tr" package.  In the instances where we call the
Errorf() function, the consequence is that we now pass the format string
and any additional arguments to the Get() method, and then just pass
the string it returns to the Errorf() function as a single argument.

As of Go 1.24, if the Go version number in the "go.mod" file is set to
1.24 or higher, the "go vet" command now reports misuses of non-constant
strings as format strings.  This includes all the cases where we pass
the output from the Get() method of the Tr structure to an Errorf()
function as a single argument.  To resolve this concern, we revise
all such instances to use the New() function of our "errors" package
in place of the Errorf() function.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Mar 5, 2025
When we report an error, we often call the Wrapf() function in our
custom "errors" package, which accepts an existing error, a format
string, and a variable number of additional arguments.  The intent of
this function is for the additional arguments to be interpolated into
the format string, and then a new error created using the string which
results from the interpolation, combined with the initial error and
its message.

In many instances we originally made use of the interpolation capability
of the Wrapf() function by passing at least one additional argument to
it other than the existing error and the format string.  In other cases,
though, we used the Wrapf() function with just the existing error and
a fixed string argument, so the Wrap() function of the same "errors"
package would have sufficed instead.

In PR git-lfs#4781 updated our code to allow for localization of most of our
output messages by inserting calls to the Get() method of the Tr
structure of our "tr" package.  In the instances where we call the
Wrapf() function, the consequence is that we now pass the format string
and any additional arguments to the Get() method, and then just pass
the string it returns to the Wrapf() function as the format string
argument following the initial existing error argument.

As of Go 1.24, if the Go version number in the "go.mod" file is set to
1.24 or higher, the "go vet" command now reports misuses of non-constant
strings as format strings.  This includes all the cases where we pass
the output from the Get() method of the Tr structure to an Wrapf()
function as a format string without any additional arguments to be
interpolated.  To resolve this concern, we revise all such instances to
use the Wrap() function of our "errors" package in place of the Wrapf()
function.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants