test3026: try to reduce runtime within legacy mingw build environments by mback2k · Pull Request #9412 · curl/curl

mback2k · 2022-09-01T20:24:40Z

Right now the test seems to take between 15 and 20 minutes
on our Windows CI environments. Which is quite extreme for
just 100 threads, but anyway 42 looks like a better number.

jay · 2022-09-02T06:55:07Z

Right now the test seems to take between 15 and 20 minutes
on our Windows CI environments.

The test should not even take 1 minute, why does it take 15?

bagder · 2022-09-02T12:48:37Z

If a 100 threads take 20 minutes, won't 42 threads then still take a rather excessive 8.4 minutes?

Maybe the test should rather be changed to have a max time for which it can execute?

mback2k · 2022-09-02T13:25:23Z

@jay not sure, on my machine it takes less than a minute. I think the CI VMs have trouble with great parallelism due to limited number of cores.

@bagder I wasn't sure if the issue scaled in a linear fashion. I thought let's try 42 and see what happens. As stated above, I think the main problem is the thread pressure on our CI hosts. Locally it works fast and fine as expected.

mback2k · 2022-09-03T17:53:27Z

As I guessed it does not seem to be a linear issue.

Before / without this PR: https://ci.appveyor.com/project/curlorg/curl/builds/44661247/job/fq5hqnkj4bjr90p5#L4987

test 3026...[curl_global_init thread-safety]
-------e--- OK (1510 out of 1510, remaining: 00:00, took 683.379s, duration: 31:14)

After / with this PR: https://ci.appveyor.com/project/curlorg/curl/builds/44653968/job/hk9etjq6tuwo0ca7#L4988

test 3026...[curl_global_init thread-safety]
-------e--- OK (1510 out of 1510, remaining: 00:00, took 87.706s, duration: 19:43

Overall this seems to cut quite some time from the AppVeyor CI runs which run tests.

Before vs. After cmake builds (top 4 MSVC, bottom 4 MSYS/MinGW):

Before vs. After autotools builds (3 MSYS/MinGW and 1 CygWin):

Seems like this is primarily affecting the 4 cmake-based MSYS/MinGW in the middle.

mback2k · 2022-09-03T17:59:06Z

To sum up, only builds using the following compilers on AppVeyor are affected by the slowness after all:

C:\mingw-w64\x86_64-8.1.0-posix-seh-rt_v6-rev0\mingw64\bin
C:\mingw-w64\x86_64-7.2.0-posix-seh-rt_v5-rev1\mingw64\bin
C:\mingw-w64\i686-6.3.0-posix-dwarf-rt_v5-rev1\mingw32\bin
C:\MinGW\bin

So I guess it is their threading model which is causing the trouble. @MarcelRaad any suggestions?

Edit: confirmed also by Azure CI running classic MinGW and similar variants:

bagder · 2022-09-29T11:56:16Z

This is still set a draft but wants a review. Isn't that contradictory?

mback2k · 2022-10-08T22:11:14Z

I will need to reproduce this locally and figure out a better way to shorten the runtime for legacy MinGW builds.

mback2k · 2022-10-11T11:47:05Z

Update: I was able to reproduce this locally with classic/legacy MinGW. Next up is identifying the root cause / finding workaround.

mback2k · 2022-10-11T20:59:38Z

Root cause: the slow part is loading of iphlpapi.dll (or any other library) as part of Curl_win32_init during curl_global_init.

bagder · 2022-11-14T08:08:02Z

Should we just close this?

jay · 2022-11-14T09:10:20Z

Root cause: the slow part is loading of iphlpapi.dll (or any other library) as part of Curl_win32_init during curl_global_init.

I pushed a change to your branch that calls loadlibrary on secur32 and iphlpapi in the main thread. This way each time curl_global_init/cleanup is called from the worker threads and it calls loadlibrary/freelibrary on those two libraries it should just increase/decrease their refcount. This may or may not work depending on if initialization of the library is taking a long time, and not the actual loading.

mback2k · 2022-11-14T18:40:33Z

@jay thanks, it is definitely the loading part according to my tests. But even reversing the binaries and running them outside of msys-shells (v1 or v2) did not help me identify the difference between the various fast (mingw-w64 from msys2) and slow (any mingw on msys1) build variants.

mback2k · 2022-11-14T22:16:53Z

@jay to resolve the merge conflict which is blocking Azure Pipelines and AppVeyor you may delete my remaining commit.

- Load Windows system libraries secur32 and iphlpapi beforehand, so that libcurl's repeated global init/cleanup only increases/decreases the library's refcount rather than causing it to load/unload. Assisted-by: Marc Hoersken Closes #xxxx

jay · 2022-11-17T06:21:29Z

job windows msys v1_mingw32 now shows:

2022-11-16T07:40:24.7492812Z test 3026...[curl_global_init thread-safety]
2022-11-16T07:40:25.0123541Z -------e--- OK (1526 out of 1530, remaining: 00:02, took 0.268s, duration: 17:07)

Does that mean this is fixed or should I be checking a different job?

the code is already in a WIN32 section

mback2k · 2022-11-17T06:54:25Z

Thanks a lot @jay. Yeah, all of the msys v1 jobs experienced the slowness issue, so if one is fixed, all of them should be.

Will you take it completely from here?

jay · 2022-11-17T07:25:07Z

Will you take it completely from here?

Yup np. I'm going to wait on the CI for this last fixup to complete then I will land it.

mback2k · 2022-11-17T07:34:24Z

Sounds good, thanks again.

mback2k · 2022-11-17T11:40:58Z

CI looks good to me. I think this is ready for merge.

curl no longer supports old mingw. Follow-up to 3802910 curl#11625 Follow-up to 856b133 curl#9412

curl no longer supports old/legacy/classic mingw. This mitigation was addressing slow perf seen in CI with old mingw. The slow perf is not seen in current CI with supported compilers. Remove the duplicate DLL load function from libtest. It's no longer used after this patch. Current CI run times for test3026 on GHA/windows: ``` test 3026...[curl_global_init thread-safety] # mingw, CM clang-x86_64 gnutls libssh -------e--- OK (1715 out of 1738, remaining: 00:02, took 0.196s, duration: 02:55) # dl-mingw, CM 9.5.0-x86_64 schannel -------e--- OK (1554 out of 1577, remaining: 00:02, took 0.217s, duration: 02:29) # msvc, CM x64-windows schannel +examples -------e--- OK (1578 out of 1601, remaining: 00:02, took 0.205s, duration: 02:50) ``` Follow-up to 3802910 #11625 Follow-up to 856b133 #9412 Ref: #17413 Closes #17414

mback2k self-assigned this Sep 1, 2022

mback2k requested a review from bagder September 1, 2022 20:24

mback2k added the tests label Sep 1, 2022

mback2k removed the request for review from bagder September 29, 2022 21:48

mback2k force-pushed the speedup-test3026 branch 2 times, most recently from 74c6442 to c583bb1 Compare October 4, 2022 21:12

mback2k added the help wanted label Oct 24, 2022

mback2k changed the title ~~test3026: reduce number of threads to reduce total runtime~~ test3026: try to reduce test runtime within legacy mingw build environments Nov 14, 2022

mback2k changed the title ~~test3026: try to reduce test runtime within legacy mingw build environments~~ test3026: try to reduce runtime within legacy mingw build environments Nov 14, 2022

jay force-pushed the speedup-test3026 branch from e32972f to b1a0119 Compare November 14, 2022 22:54

jay force-pushed the speedup-test3026 branch from b1a0119 to 5755e3d Compare November 15, 2022 23:15

fix missing const qualifier

d19a31e

remove dupe WIN32 guard

a0dca97

the code is already in a WIN32 section

jay removed the help wanted label Nov 17, 2022

mback2k marked this pull request as ready for review November 17, 2022 11:41

jay closed this in 856b133 Nov 18, 2022

vszakats mentioned this pull request May 21, 2025

windows: reduce/stop loading DLLs at runtime #17413

Closed

3 tasks

vszakats added a commit to vszakats/curl that referenced this pull request May 22, 2025

lib3026: stop pre-loading DLLs to mitigate old mingw perf issue

004445e

curl no longer supports old mingw. Follow-up to 3802910 curl#11625 Follow-up to 856b133 curl#9412

vszakats mentioned this pull request May 22, 2025

lib3026: drop DLL pre-load perf mitigation for old mingw #17414

Closed

vszakats mentioned this pull request May 23, 2025

GHA/windows: run tests in a WinXP build #17426

Closed

Uh oh!

Conversation

mback2k commented Sep 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jay commented Sep 2, 2022

Uh oh!

bagder commented Sep 2, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mback2k commented Sep 2, 2022

Uh oh!

mback2k commented Sep 3, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mback2k commented Sep 3, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bagder commented Sep 29, 2022

Uh oh!

mback2k commented Oct 8, 2022

Uh oh!

mback2k commented Oct 11, 2022

Uh oh!

mback2k commented Oct 11, 2022

Uh oh!

bagder commented Nov 14, 2022

Uh oh!

jay commented Nov 14, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mback2k commented Nov 14, 2022

Uh oh!

mback2k commented Nov 14, 2022

Uh oh!

jay commented Nov 17, 2022

Uh oh!

mback2k commented Nov 17, 2022

Uh oh!

jay commented Nov 17, 2022

Uh oh!

mback2k commented Nov 17, 2022

Uh oh!

mback2k commented Nov 17, 2022

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

3 participants

mback2k commented Sep 1, 2022 •

edited

Loading

bagder commented Sep 2, 2022 •

edited

Loading

mback2k commented Sep 3, 2022 •

edited

Loading

mback2k commented Sep 3, 2022 •

edited

Loading

jay commented Nov 14, 2022 •

edited

Loading