Skip to content

cmake: speed up curldown processing, enable by default#12762

Closed
vszakats wants to merge 3 commits intocurl:masterfrom
vszakats:doc-followups
Closed

cmake: speed up curldown processing, enable by default#12762
vszakats wants to merge 3 commits intocurl:masterfrom
vszakats:doc-followups

Conversation

@vszakats
Copy link
Member

@vszakats vszakats commented Jan 23, 2024

  • cmake: enable BUILD_DOCS by default (this controls converting and
    installing .3 files from .md sources)

  • cmake: speed up generating .3 files by using a single command per
    directory, instead of a single command per file. This reduces external
    commands by about a thousand. (There remains some CMake logic kicking
    in resulting in 500 -one per file- external -E touch_nocreate calls.)

  • cd2nroff: add ability to process multiple input files.

  • cd2nroff: add -k option to use the source filename to form the
    output filename. (instead of the default in-file Title: line.)

Follow-up to 3f08d80
Follow-up to ea0b575 #12753
Follow-up to eefcc1b #12730

Closes #12762

@vszakats vszakats marked this pull request as draft January 23, 2024 06:46
@vszakats vszakats marked this pull request as ready for review January 23, 2024 09:09
@vszakats vszakats changed the title docs: curldown followups cmake: speed up curldown processing, enable by default Jan 23, 2024
@vszakats
Copy link
Member Author

Please double-check my broken Perl-fu.

@vszakats vszakats closed this in 2620aa9 Jan 23, 2024
@vszakats vszakats deleted the doc-followups branch January 23, 2024 23:07
vszakats added a commit to vszakats/curl that referenced this pull request Mar 27, 2024
This time limit the number of files per command to avoid exceeding
limitations of certain OS/shell envs.

Such known env is Windows with the `cmd.exe` shell, which features an
8K limit to this day.

Allowlisting `UNIX` to have no limit and using a limit of 200 for other
envs to be safe. If there is a way to detect `cmd.exe` and/or we know
which precise envs are sensitive to this, we can tweak these conditions
further.

Even with the low limit, this patch reduces external commands by 200x,
making builds much faster.

Ref: curl#13047
Ref: curl#12762 2620aa9

Closes #xxxxx
vszakats added a commit that referenced this pull request Apr 4, 2024
This time limit the number of files per command to avoid exceeding
limitations of certain OS/shell envs.

Such known env is Windows with the `cmd.exe` shell, which features an
8K command-line length limit to this day.

Allowlisting `UNIX` to have no limit and using a limit of 200 for other
envs to be safe. If there is a way to detect `cmd.exe` and/or we know
which precise envs are sensitive to this, we can tweak these conditions
further.

Even with the low limit, this patch reduces external commands by 200x,
making builds much faster.

Ref: #12762 2620aa9 (initial)
Ref: #13047 f03c856 (revert)

Reviewed-by: Daniel Stenberg
Closes #13207
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

2 participants