Improve formatting of manual page synopses#6188
Conversation
The synopsis sections of our manual pages have generally been formatted in the same way since 2014, when our first manual page was added in commit 79518af of PR git-lfs#58. The fixed portions of the Git LFS command presented in each page's synopsis is delimited with backticks, and the remainder of the command is left unformatted. Our manual page source files were originally written in Ronn and now use AsciiDoc markup instead. In both cases, text delimited with backticks is displayed in a monospace font when our pages are rendered into HTML. This in itself is consistent with the style we use throughout our manual pages, as we aim to render all literals as monospace text. However, we also render all command option flags and argument placeholder names as monospace text, except within the majority of our synopsis blocks, where these are displayed in a proportional font. There are some exceptions, such as in our git-lfs-completion(1), git-lfs-filter-process(1), and git-lfs-pointer(1) pages, where the entirety of each line of the synopsis is rendered as monospace text, which is more consistent with how we display command arguments and flags in the other sections of our manual pages. Even in these cases, though, each line of the synopsis is rendered separately, because we have not placed the entire synopsis within a literal block. To prevent the lines being displayed as a continuous single line, we add trailing hard line breaks in the form of a space character and a plus ("+") character at the end of the lines. This is a fragile syntax, though, as can be illustrated by the current state of our git-lfs-push(1) manual page, where some hard line breaks were inadvertently omitted when additional lines describing the command's new --stdin option were inserted into the page's synopsis in commit e35f407 of PR git-lfs#5086. Likewise, when we implemented a --stdin option for the "git lfs fetch" command in commit 5c6daf0 of PR git-lfs#6088, we added another line to the synopsis section of the git-lfs-fetch(1) manual page but neglected to add a hard line break as well. We can avoid these types of issues and format our synopsis sections in a manner consistent with the rest of our manual page text by using AsciiDoc source listing blocks with custom subtitutions. We place all of the command prototypes in each synopsis section into a single block, which means we can eliminate the trailing hard line breaks completely. We also remove all the backtick delimiters from our synopsis sections, since these sections will now be fully rendered as monospace text. Since commit f11d309 of PR git-lfs#5054 our "mangen" utility has recognized AsciiDoc source listing blocks and rendered them into plain text appropriately. (Note that we use the "mangen" utility to convert our manual page source files into formatted plain text encapsulated in Go string assignments, which are then output by the "git lfs help" command.) In addition to the "git lfs help" command, we also provide manual pages in HTML and roff formats, which we include in the packages we publish for each release. To generate these HTML- and roff-formatted pages we make use of the Asciidoctor "manpage" converter, which accepts our AsciiDoc source files as input. At present, though, the Asciidoctor "manpage" converter always indents source listing blocks when generating output in the roff format, so when these pages are viewed with the man(1) command, our synopsis sections will now be indented, unlike all the content of all of the other sections. As we would prefer for all of our manual pages' sections to be aligned evenly, in a subsequent commit in this PR we will introduce a small Asciidoctor extention module to prevent the indentation of source listing blocks, but only for the blocks in our synopsis sections and only when generating output in the roff format.
In a previous commit in this PR we revised all the synopsis sections of our AsciiDoc manual pages so they were formatted consistently using source listing blocks, which helps us avoid problems with missing hard line breaks, among other advantages. However, this change introduces one small but noticeable inconsistency in the way our manual pages are now displayed by the man(1) program, due to how the Asciidoctor "manpage" converter handles listing blocks when generating roff output. The "manpage" converter always indents listing and literal blocks by a fixed amount, which is currently set to four space characters, as seen in the convert_listing() and convert_literal() methods of the Converter::ManPageConverter Ruby class: https://github.com/asciidoctor/asciidoctor/blob/fc0d033577d30adbffba73ce06709292fc2cf3ce/lib/asciidoctor/converter/manpage.rb#L253-L260 https://github.com/asciidoctor/asciidoctor/blob/fc0d033577d30adbffba73ce06709292fc2cf3ce/lib/asciidoctor/converter/manpage.rb#L269-L276 These methods enclose the content of the blocks with a set of roff macros, two of which in particular indent and unindent the text. In general, this is desirable, as we expect to use source listing and literal blocks within a series of regular paragraphs, and so rendering them in an indented fashion makes them distinct and recognizable as examples of source code or other literal text. Our synopsis sections, though, explicitly do not contain any content other than Git LFS command prototypes, which we would like to align with the main content of the other sections, both for visual consistency and because this is the standard format used for all Unix manual pages. Because the roff indentation macros are, at present, hard-coded into the Asciidoctor "manpage" converter's methods, we can not override them with simple AsciiDoc options or flags. Instead, to achieve the formatting we desire for our synopsis sections, we introduce a small Asciidoctor extension module which subclasses the Converter::ManPageConverter class and provides its own definition of the convert_literal() method. In this method, we first call the parent class's method of the same name, and then modify its output to remove the roff indentation and unindentation macros, but only if the current document node's "role" attribute is set to "synopsis". To locate the indentation and unindentation macros in our extension module, we define a regular expression which matches the roff macros output by the convert_literal() method of the Converter::ManPageConverter class, which are the ".RS" ("region start") and ".RE" ("region end") indentation macros. We expect the former will be followed by an numeric value, which currently is always "4", but we allow for any possible value here in case Asciidoctor changes the indentation amount for listing blocks in the future. We also allow for these macros to be preceded by ".if" conditional requests with the "n" condition, which is how Asciidoctor generates them at present. The "n" (i.e., "nroff") condition is normally set true by default if the output device is a terminal. See, for reference: https://www.gnu.org/software/groff/manual/groff.html#Indented-regions-in-ms https://www.gnu.org/software/groff/manual/groff.html#troff-and-nroff-Modes Finally, we add a custom "role" attribute to the source listing blocks in all of our manual pages' synopsis sections, and we update our Makefile so Asciidoctor will find and load our new extension module when generating roff output with the "manpage" converter.
The documentation of our commands generally conforms to the conventions of the Unix and Git manuals. However, a number of inconsistencies are present in both the content and format of our manual pages. In a previous commit in this PR we revised all the synopsis sections of our AsciiDoc manual pages so they were formatted consistently using source listing blocks, which helps us avoid problems with missing hard line breaks, among other advantages. In a subsequent commit in this PR we expect to make additional changes so that when our manual pages are rendered into HTML or roff, either by an Asciidoctor converter or by GitHub's UI, the synopsis sections of our pages will identify literals using boldface text and replaceable arguments using italics. This will bring our documentation into closer alignment with the standards of the Linux man-pages project: https://git.kernel.org/pub/scm/docs/man-pages/man-pages.git/tree/man/man7/man-pages.7?id=a5342ef55f0a96790bf279a98c9d2a30b19fc9eb#n200 However, our "mangen" program should continue to render our manual pages into plain text, as this is the format in which they are output by the "git lfs help" command. We will therefore update this program to remove the AsciiDoc markup for boldface and italic text from the content of the synopsis sections of our manual pages. Before we make these changes, we first need to revise the synopsis sections of several manual pages whose existing format would conflict with the additional AsciiDoc markup we intend to use to identify italic text in particular. AsciiDoc uses underscore and asterisk characters to demarcate italic and boldface text, respectively. Since we expect to make use of these formatting characters, we will also have to update our "mangen" program so that it removes them from our synopsis sections. Ultimately, we intend to only use the underscore and asterisk characters in constrained pairs rather than unconstrained pairs, meaning they should only appear immediately before or after terms, never within a term: https://docs.asciidoctor.org/asciidoc/latest/text/#constrained https://docs.asciidoctor.org/asciidoc/latest/text/#unconstrained However, adjusting our "mangen" program to remove underscore and asterisk characters from synopsis sections will be substantially simpler if we do not have to account for this difference. We can just remove all such characters from our synopsis sections instead of trying to match formatting characters only when they appear between the start or end of a term and either whitespace or the start or end of a line. For this reason, we would prefer that no underscore characters appear in any terms in our synopsis sections, as they do now in our git-lfs-post-checkout(1) and git-lfs-post-merge(1) manual pages. At present, we define the replaceable arguments of those two commands with terms that contain underscore characters, such as "is_branch_checkout" and "is_squash". As a further incentive to replace the underscore characters in these two manual pages' synopsis sections, the Git project's guidelines for manual page documentation state that command examples should use only "dashes" (i.e., the ASCII "hyphen-minus" character) to separate words: https://github.com/git/git/blob/c4a0c8845e2426375ad257b6c221a3a7d92ecfda/Documentation/CodingGuidelines#L871-L878 Therefore, both to bring our documentation into closer conformance with Git's standards and to allow our changes to the "mangen" program to be as simple as possible, we change the underscore characters in the git-lfs-post-checkout(1) and git-lfs-post-merge(1) manual pages' synopsis sections to hyphen characters.
In previous commits in this PR we revised all the synopsis sections of our AsciiDoc manual pages so they are formatted consistently using source listing blocks, and ensured that no AsciiDoc inline formatting characters appear in the synopsis sections. With these changes in place we are now able to reformat our synopsis sections so that when our manual pages are rendered into HTML or roff, either by an Asciidoctor converter or by GitHub's UI, our synopses will consistently identify literals using boldface text and replaceable arguments using italics. In doing so, we will bring our documentation into closer alignment with the standards of the Linux man-pages project: https://git.kernel.org/pub/scm/docs/man-pages/man-pages.git/tree/man/man7/man-pages.7?id=a5342ef55f0a96790bf279a98c9d2a30b19fc9eb#n200 First, we apply the AsciiDoc "quotes" substitution rule to all of our synopsis sections, so that Asciidoctor will respect the AsciiDoc inline formatting characters we want to use, namely the underscore and asterisk characters for boldface and italic text, respectively. We also need to specify the "verbatim" rule, since we are overriding the default rule settings for source listing blocks, but still want to start by applying that rule: https://docs.asciidoctor.org/asciidoc/latest/subs/#verbatim-group https://docs.asciidoctor.org/asciidoc/latest/subs/quotes/ We then add the appropriate inline formatting markup to all of the synopsis sections of our manual page source files. We demarcate all literals using boldface text, and all replaceable arguments using italics. Note that in the our git-lfs-pointer(1) manual page we make use of unconstrained formatting pairs in order to render the existing synopsis text without other changes, as this page is the only one where we include the optional equals sign character between literal and replaceable arguments, e.g., "--file=path/to/file" instead of "--file path/to/file". We expect to further revise this page in a subsequent PR at which time we will also rewrite a number of other pages' synopsis sections so they use a consistent style for all of their text and placeholder arguments, and only require the use of constrained formatting pairs: https://docs.asciidoctor.org/asciidoc/latest/text/#constrained https://docs.asciidoctor.org/asciidoc/latest/text/#unconstrained Next, we revise our "mangen" program so that it continues to render our manual pages into plain text, as this is the format in which they are output by the "git lfs help" command. Specifically, we update this program to remove the AsciiDoc markup for boldface and italic text from the content of the synopsis sections of our manual pages. The "mangen" program has recognized the start and end of AsciiDoc source listing blocks since commit f11d309 of PR git-lfs#5054, when we first migrated our manual pages' source files to the AsciiDoc format. We now expand this logic so that it detects when a source listing block is defined with our custom "synopsis" role and sets an internal "isSynopsisSourceBlock" flag, which remains "true" until the block's final line is encountered. While the "isSynopsisSourceBlock" flag is set we apply a simple regular expression which removes any asterisk or underscore characters from each line. As we noted in a previous commit in this PR, this approach is significantly less complicated than if we attempted to match inline formatting characters only when they appear between the start or end of a term and either whitespace or the start or end of a line. Further, this regular expression also allows us to handle unconstrained formatting pairs, which as explained above we make use of in our git-lfs-pointer(1) manual page for now. Finally, we introduce a new t/t-usage.sh shell test script with a single initial "usage: no command specified" test which verifies that our git-lfs(1) manual page is output by the multiple variants of the "git lfs" and "git-lfs" commands. Note that we cannot test using the "git lfs -h" or "git-lfs -h" variants because at present we do not support the -h option when it is provided without a Git LFS command name, as discussed in git-lfs#6168.
larsxschneider
left a comment
There was a problem hiding this comment.
Thanks for cleaning this up! I’m always amazed by how much thought you put into every detail 🙇
| *git lfs pointer* **--file=**__path/to/file__ | ||
| *git lfs pointer* **--file=**__path/to/file__ **--pointer=**__path/to/pointer__ | ||
| *git lfs pointer* **--file=**__path/to/file__ *--stdin* | ||
| *git lfs pointer --check* **--file=**__path/to/file__ |
There was a problem hiding this comment.
**foo** prints text bold too, right? Why is * not sufficient here?
There was a problem hiding this comment.
Yes, ** can also be used to demarcate bold text, and __ can be used for italics. These are used in unconstrained pairs, while the single-character form is used in constrained pairs.
The key difference is that constrained pairs require a word boundary to be parsed correctly, so we could write *--file* _<file>_ and that would be rendered as we expect, but if we write *--file=*_<file>_, that doesn't work because the = character (and lack of any space character) effectively turns the whole phrase into a single "word".
Now this is all actually just a temporary state of affairs, I very much hope! I intend to write up another PR to make our various manual pages use consistent terms and syntax for things like placeholder arguments. Right now, we have a mix of <file>, <path>, <directory>, and in this case, path/to/file, which is very non-standard; we also mix <options> and plain options, etc.
We should also remove the = characters from this page's synopsis as we don't use them in any other manual pages, at which point we can just use constrained markup pairs instead of the ** and __ unconstrained pairs.
I thought about trying to include those changes in this PR too, but decided it was better to tackle one type of change at a time, so this PR just converts our manual pages to a new synopsis format, and the next PR can adjust the actual text of those synopses and the other related parts of the pages, like options lists.
In a prior commit in this PR we updated the printHelp() function in our "commands" package to recognize the -h option when it is provided without any Git LFS command name, which should resolve the issue reported in git-lfs#6168. (Note that we already recognized the --help option when no Git LFS command name is provided.) In the same commit we also introduced a new "commands/run_test.go" source file with a single Go test function that verified the revised behaviour of the printHelp() function. Subsequently, in commit 926dc36 of PR git-lfs#6188, we added a new "t/t-usage.sh" shell test script with a single test. This test verifies that the "git lfs" and "git-lfs" commands respond to various conditions by returning the contents of our git-lfs(1) manual page. These conditions include the use of the --help option without any Git LFS command name, e.g., when a "git lfs --help" command is run. We can therefore now expand the "usage: no command specified" shell test so that it runs both the "git lfs -h" and "git-lfs -h" commands and confirms that the git-lfs(1) manual page is returned, as should be the case due to the changes from our prior commit in this PR. The shell test is able to exercise the "git-lfs" binary after it has been compiled with a complete set of manual pages in its ManPages map, since our "Makefile" ensures that our "mangen" program is compiled and executed to generate the ManPages map and store it into a "commands/mancontent_gen.go" file. The "git-lfs" binary is then compiled from sources which include this generated file. The Go language test we introduced earlier in this PR, on the other hand, has to handle the case that the ManPages map does not contain an appropriate entry for the git-lfs(1) manual page, as well as the case where an entry already exists. As a result, the Go language test is more complicated than the two lines we can now just add to our "usage: no command specified" shell test, while also not fully testing the "git-lfs" binary with a -h command-line option and instead just testing the printHelp() function directly. We therefore defer to our shell test to validate our changes to the printHelp() function, and remove the Go test and new source file we initially added.
This PR updates the synopsis sections of our manual pages to resolve several errors and to introduce additional formatting to make our documentation more clear and accessible.
Specifically, we now format our manual pages' synopsis sections using AsciiDoc source listing blocks with custom settings that permit us to demarcate all literal text in boldface and all replaceable arguments in italics. This additional formatting will be visible whenever our manual pages are rendered in HTML or displayed by the
man(1)program from roff sources.The use of boldface for literal text and italics for placeholder arguments is common to the manual page conventions of many systems, including the style guide of the Linux
man-pages(7)manual and the BSD manual page guide.This PR will be most easily reviewed on a commit-by-commit basis.
Note that we expect to make further enhancements to our manual page documentation in the future, but we defer those changes to subsequent PRs.
Line Breaks
One major advantage to the use of source listing blocks is that in our command synopses with multiple lines, we no longer need to remember to include trailing
+characters to force breaks between the lines.For example, in PR #6088 we updated the
git-lfs-fetch(1)manual page and added a second line to the synopsis, but accidentally did not add a trailing+character to the first line, so at present the GitHub UI renders both lines of the page's synopsis consecutively on a single line:By using source listing blocks for our synopses, we ensure that problems of this type can not occur again.
Rendered Examples
The synopsis of our
git-lfs-checkout(1)command's manual page renders in the GitHub UI as shown below, first without the changes from this PR and then with them. Notice that at present two separate fonts are used, one monospace and the other a proportional-width font, and only the text in the monospace font has a grey background:The revisions made in this PR ensure that the entire section appears in monospace with a common grey background, as well as distinguishing the literal text and placeholders both from each other and from the syntactic characters such as
[,{, and|.The same manual page's synopsis is rendered by the
man git-lfs-checkoutcommand on a macOS system as shown below, first without the changes from this PR and then with them:Note that the indentation remains the same, while boldface and underlined text demarcates the literals and placeholders, respectively.
Asciidoctor Extension
One inconvenient consequence of the use of AsciiDoc source listing blocks for our manual page synopses is that the Asciidoctor
manpageconverter we use to generate copies of our manual pages in the roff format expected by theman(1)program always indents these blocks. This results in an extra level of indentation, as shown below using theman git-lfs-envcommand as an example (the red arrow highlights the excess indentation):Unfortunately, it is not possible at present to change this aspect of the
manpageconverter's behaviour via a configuration option. Instead, we add a customrole=synopsisattribute to the source blocks in our synopsis sections, and introduce a small Asciidoctor extension program which removes the extra indentation from the roff formatting generated by the converter, but only when the block has therole=synopsisattribute defined. The result is that our synopsis sections are aligned with all the other sections of our manual pages, as shown below:Plain Text Output
When a user runs the
git lfs helpcommand, at present we do not pass through the output of aman(1)command in the same manner as thegit helpcommand, but instead output a plain-text version of the appropriate Git LFS manual page from an internal map compiled into thegit-lfsbinary itself. This map is generated during our build process by our standalonemangenprogram, which attempts to remove all AsciiDoc formatting from our manual page source files while still retaining their basic structure.Because our AsciiDoc manual page source files now include inline formatting markup within their synopsis sections, we revise the
mangenprogram to remove this markup, specifically the*and_characters.Fortunately there are no instances of the
*character in any of our synopses, and the_character only appears in variable argument names in two of our manual pages, thegit-lfs-post-checkout(1)page and and thegit-lfs-post-merge(1)page.Since there is no semantic difference between the use of underscore characters to separate words in argument names and the use of hyphen (
-) characters, and the latter are both more consistent with our other pages and are also recommended by the Git documentation style guide, we simply replace the underscores with hyphens in the few instances where they appear in argument names.We also introduce a new
t/t-usage.shshell test script with a single initial test to verify that ourmangenprogram correctly removes the AsciiDoc formatting from the synopsis sections of our manual pages. (We expect to later enhance this test when we merge the changes from PR #6172.)Unrelated CI Failures
Note that our
Build with latest GitCI jobs are currently failing due to an unrelated issue caused by a recent change to Git'smasterbranch, specifically that from commit git/git@3e2836a. I've advised the Git mailing list of this issue and will await feedback from the upstream Git developers on whether we should or should not revise our tests to accommodate this change.