tool: Implement non-blocking STDIN read on Windows#17572
tool: Implement non-blocking STDIN read on Windows#17572denandz wants to merge 23 commits intocurl:masterfrom
Conversation
Implements a seperate read thread for STDIN on Windows when curl is run with -T/--upload-file . This uses a similar technique to the nmap/ncat project, spawning a seperate thread which creates a loop-back bound socket, sending STDIN into this socket, and reading from the other end of said TCP socket in a non-blocking way in the rest of curl. Closes curl#17451
This comment was marked as outdated.
This comment was marked as outdated.
Co-authored-by: Viktor Szakats <vszakats@users.noreply.github.com>
Fix implicit TerminateThread call
|
Alright! This seems to be building / testing successfully now. Looks like the two fails on macOS are due to test suites timing out if I'm reading the log correctly. Not sure if that's related to my changes? I think this PR is ready for review,. |
|
Does #17569 work for you? |
Hi Jay. That pull request is for a different issue (performance of To answer your question though - Yes, I see improved performance under linux with #17569 and even further improved performance with #17566 - but that's not what this PR is about. Stdin performance issue is #17553 |
|
Thanks team! Let me know if there is anything else you need from my end to get this merged |
|
Thanks! |
Implements a seperate read thread for STDIN on Windows when curl is run with -T/--upload-file . This uses a similar technique to the nmap/ncat project, spawning a seperate thread which creates a loop-back bound socket, sending STDIN into this socket, and reading from the other end of said TCP socket in a non-blocking way in the rest of curl. Fixes curl#17451 Closes curl#17572
Replace `_beginthreadex()` C runtime calls with native win32 API `CreateThread()`. The latter was already used in `src/tool_doswin.c` and in UWP and Windows CE builds before this patch. After this patch all Windows flavors use it. To drop PP logic and simplify code. While working on this it turned out that `src/tool_doswin.c` calls `TerminateThread()`, which isn't recommended by the documentation, except for "the most extreme cases". This patch makes no attempt to change that code. Ref: 9a26633 #17572 Ref: https://learn.microsoft.com/windows/win32/api/processthreadsapi/nf-processthreadsapi-terminatethread Also: - use `WaitForSingleObjectEx()` on all desktop Windows. Ref: 4be80d5 Ref: https://sourceforge.net/p/curl/feature-requests/82/ Ref: https://learn.microsoft.com/windows/win32/api/synchapi/nf-synchapi-waitforsingleobjectex - tests: drop redundant casts. - lib3207: fix to not rely on thread macros when building without thread support. Assisted-by: Jay Satiro Assisted-by: Marcel Raad Assisted-by: Michał Petryka Follow-up to 3802910 #11625 Closes #18451
Replace `WSASocketW()` with `CURL_SOCKET()`. Also replace a call to `socketclose()` with `sclose()`. According to a comment, `socketclose()` was chosen to silence test 1498 (and 2300) reporting `MEMORY FAILURE`. These reports were accurate, and were caused by calling `WSASocketW()` instead of `socket()` (now `CURL_SOCKET()`). This also fixes the curl `sclose()` call on an error branch, which is now correctly paired with a curl socket open. The mismatched open/close calls caused an issue in TrackMemory-enabled (aka `CURLDEBUG`) builds. Docs confirm that `socket()` is defaulting to overlapped I/O, matching the replaced `WSASocketW()` call: https://learn.microsoft.com/windows/win32/api/winsock2/nf-winsock2-socket#remarks Also: - checksrc: ban `WSASocket*()` functions. - report `SOCKERRNO` instead of `GetLastError()` for socket calls, to match the rest of the codebase. Follow-up to 9a26633 #17572 Closes #18633
- fix duplicate space in error message (UWP) Follow-up to 9a26633 curl#17572
Follow-up to a81ab3e curl#20452 Follow-up to 9a26633 curl#17572
Follow-up to a81ab3e curl#20452 Follow-up to 9a26633 curl#17572
For general readability. Also to match the rest of the source code. - bump `send()` result type from `int` to `ssize_t`. - fix an `int` to be `curl_socklen_t`. - `.S_un.S_addr` -> `.s_addr`. - `SD_RECEIVE` -> `SHUT_RD`. - `SD_SEND` -> `SHUT_WR`. Follow-up to a81ab3e #20452 Follow-up to 9a26633 #17572 Closes #20457
Follow-up to 9a26633 curl#17572
Follow-up to 9a26633 curl#17572
Implements a seperate read thread for STDIN on Windows when curl is run with -T/--upload-file .
This uses a similar technique to the nmap/ncat project, spawning a seperate thread which creates a loop-back bound socket, sending STDIN into this socket, and reading from the other end of said TCP socket in a non-blocking way in the rest of curl.
Fixes #17451