Skip to content

push: add ability to read refs/oids from stdin#5086

Merged
bk2204 merged 1 commit intogit-lfs:mainfrom
rcoup:rc-push-oids-stdin
Aug 17, 2022
Merged

push: add ability to read refs/oids from stdin#5086
bk2204 merged 1 commit intogit-lfs:mainfrom
rcoup:rc-push-oids-stdin

Conversation

@rcoup
Copy link
Contributor

@rcoup rcoup commented Aug 12, 2022

Add the ability to provide refs or object ids to git lfs push via standard input. This can make life simpler when plumbing together commands or working with large numbers of OIDs/refs. Particularly on Windows where the maximum command line length is much shorter.

Read refs from stdin:

echo "mybranch" | git lfs push origin --stdin

Read object IDs from stdin:

printf "4c48d2a6991c9895bcddcf027e1e4907280bcf21975492b1afbade396d6a3340\n82be50ad35070a4ef3467a0a650c52d5b637035e7ad02c36652e59d01ba282b7\n\n" \
  | git lfs push origin --object-id --stdin

Values read from stdin are newline-delimited, and blank lines are ignored. If --stdin is used, any values passed via the command line arguments are ignored raise an error.

@bk2204
Copy link
Member

bk2204 commented Aug 15, 2022

Can you help me understand why xargs isn't sufficient here? It seems like that should work just fine in most cases, and I know it's included with Git for Windows, so it's available even on Windows.

@rcoup
Copy link
Contributor Author

rcoup commented Aug 15, 2022

Use case: safely setting up piping via xargs is annoying when git-lfs is being called from Python.

Copy link
Member

@bk2204 bk2204 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a reasonable feature to add. I had one request for an improvement.

Read refs from stdin:
`printf "branch\ntag" | git lfs push <remote> --stdin`

Read object IDs from stdin:
`printf "c0ffee..." | git lfs push --object-id <remote> --stdin`

Values read from stdin are newline-delimited.
@rcoup rcoup force-pushed the rc-push-oids-stdin branch from a7c41fb to e35f407 Compare August 17, 2022 09:52
Copy link
Member

@bk2204 bk2204 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me. Thanks again for the patch.

@bk2204 bk2204 merged commit c6497f8 into git-lfs:main Aug 17, 2022
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request May 15, 2025
The synopsis sections of our manual pages have generally been formatted
in the same way since 2014, when our first manual page was added in
commit 79518af of PR git-lfs#58.  The fixed
portions of the Git LFS command presented in each page's synopsis is
delimited with backticks, and the remainder of the command is left
unformatted.

Our manual page source files were originally written in Ronn and now
use AsciiDoc markup instead.  In both cases, text delimited with backticks
is displayed in a monospace font when our pages are rendered into HTML.
This in itself is consistent with the style we use throughout our manual
pages, as we aim to render all literals as monospace text.

However, we also render all command option flags and argument placeholder
names as monospace text, except within the majority of our synopsis
blocks, where these are displayed in a proportional font.

There are some exceptions, such as in our git-lfs-completion(1),
git-lfs-filter-process(1), and git-lfs-pointer(1) pages, where the entirety
of each line of the synopsis is rendered as monospace text, which is more
consistent with how we display command arguments and flags in the other
sections of our manual pages.

Even in these cases, though, each line of the synopsis is rendered
separately, because we have not placed the entire synopsis within
a literal block.  To prevent the lines being displayed as a continuous
single line, we add trailing hard line breaks in the form of a space
character and a plus ("+") character at the end of the lines.

This is a fragile syntax, though, as can be illustrated by the current
state of our git-lfs-push(1) manual page, where some hard line breaks
were inadvertently omitted when additional lines describing the command's
new --stdin option were inserted into the page's synopsis in commit
e35f407 of PR git-lfs#5086.

We can avoid these types of issues and format our synopsis sections
in a manner consistent with the rest of our manual page text by using
AsciiDoc source listing blocks with custom subtitutions.  We place
all of the command prototypes in each synopsis section into a single
block, which means we can eliminate the trailing hard line breaks
completely.

By using source listing blocks we can support two additional features
which should improve the readability of our manual pages' synopses.
First, we can apply the "quotes" substitution rule, which means we
can format the fixed portions of each command line such that they will
be displayed as bold text when rendered into HTML.  This formatting
aligns with the convention seen used by many Unix manual pages, and
also those of the Git project.

The other feature we can now easily make use of in our synopsis blocks
is the ability to break a single command prototype onto multiple lines,
by indenting the second and subsequent lines so they are clearly
identifiable as parts of a single prototype.  In several subsequent
commits in this PR we will revise some command prototypes so they
are long enough that they will need to be split onto multiple lines,
at which point the ability to indent lines within a single listing
block will be highly advantageous.

Since commit f11d309 of PR git-lfs#5054
our "mangen" utility has recognized AsciiDoc source listing blocks
and rendered them into plain text appropriately.  (Note that we use the
"mangen" utility to convert our manual page source files into formatted
plain text encapsulated in Go string assignments, which are then output
by the "git lfs help" command.)

We expect that in a subsequent commit in this PR we will modify the
"mangen" program to ignore the asterisk character when it appears at
the start or end of a Git LFS command in the new source listing blocks
in our synopsis sections, as for the moment these asterisks will be
rendered literally in the plain text output.

As well, we also expect that in another commit in this PR we will
introduce a small Asciidoctor extention module that will remove the
indentation the current Asciidoctor "manpage" converter always sets
for source listing blocks, but only for the blocks in our synopsis
sections.  This should prevent the synopsis prototypes from appearing
indented by an extra amount when we render our manual page source
files into the roff format.
larsxschneider added a commit that referenced this pull request Aug 18, 2025
Replace the remaining os.Exit() calls with our Exit() wrapper for
consistency. This change also alters the exit code for these exits from
1 to 2. This is irrelevant for the fetch code as it was not released,
yet. The push code was shipped some time ago (#5086)
and therefore we need to mention this change in the release notes.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Jan 12, 2026
The synopsis sections of our manual pages have generally been formatted
in the same way since 2014, when our first manual page was added in
commit 79518af of PR git-lfs#58.  The fixed
portions of the Git LFS command presented in each page's synopsis is
delimited with backticks, and the remainder of the command is left
unformatted.

Our manual page source files were originally written in Ronn and now
use AsciiDoc markup instead.  In both cases, text delimited with backticks
is displayed in a monospace font when our pages are rendered into HTML.
This in itself is consistent with the style we use throughout our manual
pages, as we aim to render all literals as monospace text.

However, we also render all command option flags and argument placeholder
names as monospace text, except within the majority of our synopsis
blocks, where these are displayed in a proportional font.

There are some exceptions, such as in our git-lfs-completion(1),
git-lfs-filter-process(1), and git-lfs-pointer(1) pages, where the entirety
of each line of the synopsis is rendered as monospace text, which is more
consistent with how we display command arguments and flags in the other
sections of our manual pages.

Even in these cases, though, each line of the synopsis is rendered
separately, because we have not placed the entire synopsis within
a literal block.  To prevent the lines being displayed as a continuous
single line, we add trailing hard line breaks in the form of a space
character and a plus ("+") character at the end of the lines.

This is a fragile syntax, though, as can be illustrated by the current
state of our git-lfs-push(1) manual page, where some hard line breaks
were inadvertently omitted when additional lines describing the command's
new --stdin option were inserted into the page's synopsis in commit
e35f407 of PR git-lfs#5086.

Likewise, when we implemented a --stdin option for the "git lfs fetch"
command in commit 5c6daf0 of PR git-lfs#6088,
we added another line to the synopsis section of the git-lfs-fetch(1)
manual page but neglected to add a hard line break as well.

We can avoid these types of issues and format our synopsis sections
in a manner consistent with the rest of our manual page text by using
AsciiDoc source listing blocks with custom subtitutions.  We place
all of the command prototypes in each synopsis section into a single
block, which means we can eliminate the trailing hard line breaks
completely.  We also remove all the backtick delimiters from our
synopsis sections, since these sections will now be fully rendered as
monospace text.

Since commit f11d309 of PR git-lfs#5054
our "mangen" utility has recognized AsciiDoc source listing blocks
and rendered them into plain text appropriately.  (Note that we use the
"mangen" utility to convert our manual page source files into formatted
plain text encapsulated in Go string assignments, which are then output
by the "git lfs help" command.)

In addition to the "git lfs help" command, we also provide manual pages
in HTML and roff formats, which we include in the packages we publish
for each release.  To generate these HTML- and roff-formatted pages we
make use of the Asciidoctor "manpage" converter, which accepts our
AsciiDoc source files as input.

At present, though, the Asciidoctor "manpage" converter always indents
source listing blocks when generating output in the roff format, so when
these pages are viewed with the man(1) command, our synopsis sections
will now be indented, unlike all the content of all of the other sections.

As we would prefer for all of our manual pages' sections to be aligned
evenly, in a subsequent commit in this PR we will introduce a small
Asciidoctor extention module to prevent the indentation of source
listing blocks, but only for the blocks in our synopsis sections and
only when generating output in the roff format.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants