Skip to content

GHA/codeql: add tweak to successfully build libtests for CodeQL#19632

Closed
vszakats wants to merge 5 commits intocurl:masterfrom
vszakats:cqlt2
Closed

GHA/codeql: add tweak to successfully build libtests for CodeQL#19632
vszakats wants to merge 5 commits intocurl:masterfrom
vszakats:cqlt2

Conversation

@vszakats
Copy link
Member

@vszakats vszakats commented Nov 21, 2025 •
edited
Loading

Turns out the cause of CodeQL hangs (or probably just extreme long
compile) is the header curl/typecheck-gcc.h. By accident I noticed
that the preprocessed output of libtests.c is 75 MB (megabytes). This
is much higher than the amounf of source code hinted, also compared to
e.g. units.c or other build targets. The reason for the extreme size
is each easy option call pulling in the large checker logic defined
in this header.

By compiling with -DCURL_DISABLE_TYPECHECK, preprocessed output drops
to 2.2 MB (34x), and the libtests target builds without issues.

Also build all tests and examples with the Linux HTTP/3 config, covering
3 more files.

With these, CodeQL C coverage is 893 out of 930 (96%) (was: 645 69%)

Follow-up to 71fc11e #18695
Follow-up to a333fd4 #18557
Follow-up to b4922b1 #18564

Closes vszakats#11

@vszakats vszakats marked this pull request as draft November 21, 2025 01:44
@github-actions github-actions bot added the CI Continuous Integration label Nov 21, 2025
@vszakats vszakats changed the title GHA/codeql: try something for libtests GHA/codeql: add tweak to successfully build libtests for CodeQL Nov 21, 2025
@vszakats vszakats marked this pull request as ready for review November 21, 2025 02:05
@vszakats vszakats marked this pull request as draft November 21, 2025 02:10
@vszakats vszakats marked this pull request as ready for review November 21, 2025 02:11
@vszakats vszakats closed this in fdacf34 Nov 21, 2025
@vszakats vszakats deleted the cqlt2 branch November 21, 2025 02:54
@vszakats
Copy link
Member Author

vszakats commented Nov 21, 2025

Went up to 896/930 after merging to master.

List of C files not tested by CodeQL:

docs/examples/crawler.c
docs/examples/ephiperfifo.c
docs/examples/evhiperfifo.c
docs/examples/ghiper.c
docs/examples/hiperfifo.c
docs/examples/htmltidy.c
docs/examples/multi-uv.c
docs/examples/smooth-gtk-thread.c
docs/examples/xmlstream.c
include/curl/stdcheaders.h
include/curl/typecheck-gcc.h
lib/config-mac.h
lib/config-os400.h
lib/config-plan9.h
lib/config-riscos.h
lib/config-win32.h
lib/dllmain.c
lib/setup-os400.h
lib/setup-vms.h
packages/OS400/ccsidcurl.c
packages/OS400/ccsidcurl.h
packages/OS400/curlcl.c
packages/OS400/curlmain.c
packages/OS400/os400sys.c
packages/OS400/os400sys.h
packages/vms/curl_crtl_init.c                                                                      
packages/vms/curlmsg_vms.h                                                                         
packages/vms/curlmsg.h                                                                             
packages/vms/report_openssl_version.c                                                              
packages/vms/vms_eco_level.h                                                                       
scripts/schemetable.c                                                                              
tests/cmake/test.c                                                                                 
tests/http/testenv/mod_curltest/mod_curltest.c

These are either impossible to build, and/or safe to ignore.

@vszakats vszakats mentioned this pull request Nov 21, 2025
Closed
23 tasks
@vszakats
Copy link
Member Author

vszakats commented Nov 21, 2025 •
edited
Loading

Disabling typecheck also makes building libtests much faster on the local machine. It's been a while I was wondering why it's noticeably slow compared to everything else. Probably since fd2ca23 #17955, because I'm mostly compiling with clang.

Type checking is of course nice, if not essential, but it'd need to be enabled in just a couple of CI jobs to catch mistakes, and may make the rest faster too.

(Also important because I almost put myself on a track to start parallelizing libtests builds, and that'd be a great pain.)

Disable it with:

  • autotools: CPPFLAGS=-DCURL_DISABLE_TYPECHECK
  • cmake: -DCMAKE_C_FLAGS=-DCURL_DISABLE_TYPECHECK

edit: also reminds me of a so far parked experiment to introduce a type-safe setopt API. Just one for long options would be nice.

@vszakats vszakats mentioned this pull request Nov 21, 2025
Closed
vszakats added a commit that referenced this pull request Nov 21, 2025
@vszakats
build: add build-level CURL_DISABLE_TYPECHECK options
9e6f1c5 
Usage:
- autotools: `--disable-typecheck` (or `--enable-typecheck` (default))
- cmake: `-DCURL_DISABLE_TYPECHECK=ON`.

To disable `curl_easy_setopt()`/`curl_easy_getinfo()` type checking with
supported (new) gcc and clang compilers. It is useful to improve build
performance for the `tests/libtest` target. In particular the CodeQL
analyzer may take above an hour to compile with type checking enabled,
and disabling it brings it down to seconds. On local machines it may
also cut build times in half when build testdeps, depending on platform
and compiler.

Other than these cases, we recommend leaving type checking enabled.

Ref: fdacf34 #19632

Also:
- GHA/codeql: use it.
- test1165: check in `include/curl`.
- lib1912: delete stray todo comment.
- spelling and comment nits.

Closes #19637
ading2210 pushed a commit to ading2210/curl that referenced this pull request Dec 1, 2025
@vszakats @ading2210
build: add build-level CURL_DISABLE_TYPECHECK options
7d4580e 
Usage:
- autotools: `--disable-typecheck` (or `--enable-typecheck` (default))
- cmake: `-DCURL_DISABLE_TYPECHECK=ON`.

To disable `curl_easy_setopt()`/`curl_easy_getinfo()` type checking with
supported (new) gcc and clang compilers. It is useful to improve build
performance for the `tests/libtest` target. In particular the CodeQL
analyzer may take above an hour to compile with type checking enabled,
and disabling it brings it down to seconds. On local machines it may
also cut build times in half when build testdeps, depending on platform
and compiler.

Other than these cases, we recommend leaving type checking enabled.

Ref: fdacf34 curl#19632

Also:
- GHA/codeql: use it.
- test1165: check in `include/curl`.
- lib1912: delete stray todo comment.
- spelling and comment nits.

Closes curl#19637
ading2210 pushed a commit to ading2210/curl that referenced this pull request Dec 1, 2025
@vszakats @ading2210
build: add build-level CURL_DISABLE_TYPECHECK options
f3beb47 
Usage:
- autotools: `--disable-typecheck` (or `--enable-typecheck` (default))
- cmake: `-DCURL_DISABLE_TYPECHECK=ON`.

To disable `curl_easy_setopt()`/`curl_easy_getinfo()` type checking with
supported (new) gcc and clang compilers. It is useful to improve build
performance for the `tests/libtest` target. In particular the CodeQL
analyzer may take above an hour to compile with type checking enabled,
and disabling it brings it down to seconds. On local machines it may
also cut build times in half when build testdeps, depending on platform
and compiler.

Other than these cases, we recommend leaving type checking enabled.

Ref: fdacf34 curl#19632

Also:
- GHA/codeql: use it.
- test1165: check in `include/curl`.
- lib1912: delete stray todo comment.
- spelling and comment nits.

Closes curl#19637
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

CI Continuous Integration performance tests

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@vszakats