Skip to content

core: Add support for retrieving CURL error messages, handle unexpected CURL return code on macOS, and log such codes in tests (fixes #519).#517

Merged
LinZhihao-723 merged 20 commits into
y-scope:mainfrom
LinZhihao-723:networkreader_unittest
Aug 22, 2024

Conversation

@LinZhihao-723

@LinZhihao-723 LinZhihao-723 commented Aug 15, 2024

Copy link
Copy Markdown
Member

Description

This PR fixes #519.
In recent workflows, we found that some macOS build failed because of the unit test failures. By further investigation, it is because HTTP error code 416 is not handled as CURL_HTTP_RETURNED_ERROR in the libcurl version of the GH macOS runner. The same issue won't be reproduced in ubuntu 20.04 or 22.04 even with the same version of libcurl. Therefore, this PR adds special handling for macOS to address the failure in the unit test cases.
In addition, as we were approaching the cause of the failure, we realized the CURL error message could be helpful. Therefore, we also add support for retrieving CURL error messages through the CurlDownloadHandler to improve the debugability of our CURL utilities. In the unit test, we add logs to print out the error messages when the CURL return code doesn't match the expected one.

Validation performed

  • Ensure workflow can be successfully built except on centOS, which is tracked by cent OS image fails to build #521 and is not related to this PR.
  • Unit tests all passed.

Comment thread components/core/tests/test-NetworkReader.cpp Outdated
"Unexpected CURL error code: " + std::to_string(actual)
+ "; expected: " + std::to_string(expected)
};
FAIL(error_message);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on https://curl.se/libcurl/c/libcurl-errors.html, can we also use CURLOPT_ERRORBUFFER so that we can get more details about the error (e.g., what specific HTTP error >= 400 it was)?

Comment thread components/core/tests/test-NetworkReader.cpp Outdated
Comment thread components/core/tests/test-NetworkReader.cpp Outdated
Comment thread components/core/tests/test-NetworkReader.cpp Outdated
Comment thread components/core/tests/test-NetworkReader.cpp Outdated
Comment thread components/core/src/clp/NetworkReader.hpp Outdated
Comment thread components/core/src/clp/NetworkReader.cpp Outdated
Comment thread components/core/src/clp/CurlDownloadHandler.hpp Outdated
Comment thread components/core/src/clp/CurlDownloadHandler.hpp Outdated
Comment thread components/core/src/clp/CurlDownloadHandler.cpp Outdated
Comment thread components/core/src/clp/CurlDownloadHandler.cpp Outdated
LinZhihao-723 and others added 2 commits August 21, 2024 20:10
Co-authored-by: kirkrodrigues <2454684+kirkrodrigues@users.noreply.github.com>
@LinZhihao-723 LinZhihao-723 changed the title core: Add logging in NetworkReader unit tests on unexpected CURL return code. core: Add logging in NetworkReader unit tests on unexpected CURL return code (fixes #519). Aug 22, 2024

@kirkrodrigues kirkrodrigues left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the PR title, how about:

core: Add support for retrieving CURL error messages, handle unexpected CURL return code on macOS, and log such codes in tests (fixes #519).

@kirkrodrigues

Copy link
Copy Markdown
Member

Can you:

  • Update the PR description
  • Comment on the workflow failures once they finish

@LinZhihao-723 LinZhihao-723 changed the title core: Add logging in NetworkReader unit tests on unexpected CURL return code (fixes #519). core: Add support for retrieving CURL error messages, handle unexpected CURL return code on macOS, and log such codes in tests (fixes #519). Aug 22, 2024
@LinZhihao-723

Copy link
Copy Markdown
Member Author

centOS build failed due to issue #521 which is not related to this PR.

@LinZhihao-723 LinZhihao-723 merged commit 54b832c into y-scope:main Aug 22, 2024
jackluo923 pushed a commit to jackluo923/clp that referenced this pull request Dec 4, 2024
…ed CURL return code on macOS, and log such codes in tests (fixes y-scope#519). (y-scope#517)

Co-authored-by: kirkrodrigues <2454684+kirkrodrigues@users.noreply.github.com>
@LinZhihao-723 LinZhihao-723 deleted the networkreader_unittest branch June 5, 2025 14:59
junhaoliao pushed a commit to junhaoliao/clp that referenced this pull request May 17, 2026
…ed CURL return code on macOS, and log such codes in tests (fixes y-scope#519). (y-scope#517)

Co-authored-by: kirkrodrigues <2454684+kirkrodrigues@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

network_reader_illegal_offset unit test failure on macOS

2 participants