-
Notifications
You must be signed in to change notification settings - Fork 4.4k
Compensate for Windows filesystems lacking junction support #27598
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Compensate for Windows filesystems lacking junction support #27598
Conversation
virtiofs lacking junction support
a1d2b8a to
aaae579
Compare
|
I'm supportive of this PR, but note that #17192 appears to be a different issue (that's an error in |
aaae579 to
093a02e
Compare
virtiofs lacking junction support**Problem:** Bazel fails completely on Windows when using filesystems that don't support junction/reparse point operations (e.g., virtiofs, VirtualBox shared folders, network drives, RAM disks). The fatal error occurs when `ReadSymlinkOrJunction` fails during path resolution (e.g., when Starlark code calls `.realpath`): "Cannot read link: DeviceIoControl: Incorrect function" This causes build analysis to abort completely. Additionally, `CreateJunction` failures when creating convenience symlinks produce cryptic error messages, though these were already non-fatal warnings. Both fail because `DeviceIoControl` returns `ERROR_INVALID_FUNCTION` when the filesystem doesn't implement `FSCTL_GET_REPARSE_POINT` or `FSCTL_SET_REPARSE_POINT` operations. **Solution:** Handle `ERROR_INVALID_FUNCTION` gracefully by treating it as a "not supported" condition rather than a fatal error: 1. in `ReadSymlinkOrJunction` (`file.cc`:592): return `kNotALink` instead of `kError` when `ERROR_INVALID_FUNCTION` occurs. This allows path resolution to continue for non-symlink paths on unsupported filesystems. 2. in `CreateJunction` (`file.cc`:461): return new `kNotSupported` result code when `ERROR_INVALID_FUNCTION` occurs. This produces clearer "filesystem does not support junctions" warnings instead of cryptic "Incorrect function" messages. This improves UX but doesn't change behavior (these failures were already non-fatal). This follows the try-first, fallback-on-error pattern (EAFP) used by other major projects when handling unsupported filesystem operations. **Prior art:** - Rust (rust-lang/rust#138133): checks `ERROR_INVALID_FUNCTION`, `ERROR_NOT_SUPPORTED`, and `ERROR_INVALID_PARAMETER` for filesystem operation fallbacks in `std::fs::rename`. - Microsoft STL (microsoft/STL#2077): handles junctions and reparse point errors including `ERROR_INVALID_PARAMETER` with robust fallback logic in `filesystem.cpp`. - Go (golang/go#20506): uses fallback strategies when symlink APIs are unavailable on different Windows versions. - WinFsp (winfsp/winfsp#88): documents that `ERROR_INVALID_FUNCTION` indicates `STATUS_NOT_IMPLEMENTED` for unsupported operations. - Microsoft Learn: recommends checking `FILE_SUPPORTS_REPARSE_POINTS` flag via `GetVolumeInformation`, but try-catch approach is simpler and handles edge cases where detection succeeds but operations fail. **Impact:** - enables Bazel builds on virtiofs, VirtualBox shared folders, RAM disks, and other filesystems that don't support Windows junction operations. - convenience symlinks (bazel-bin, bazel-out, etc.) still won't be created, but now with clearer error messages. **Limitations:** Full junction support would require filesystem-level changes (e.g., virtiofs driver improvements). **Testing:** Tested on Windows 11 VM with host directory mounted via virtiofs, with [rules_pkg](https://github.com/bazelbuild/rules_pkg/blob/6cdaba69ee76463b2b8e97e8d243dbb6115c3aee/toolchains/git/git_configure.bzl#L40). Before change: build analysis aborted with "Cannot read link" fatal error. After change: builds complete successfully with clearer warnings about unsupported junctions for convenience symlinks.
093a02e to
35d0faa
Compare
|
@bazel-io fork 9.0.0 |
**Problem**: Bazel fails completely on Windows when using filesystems that don't support junction/reparse point operations (e.g., virtiofs, VirtualBox shared folders, network drives, RAM disks). The fatal error occurs when `ReadSymlinkOrJunction` fails during path resolution (e.g., when Starlark code calls `.realpath`): "Cannot read link: DeviceIoControl: Incorrect function". This causes build analysis to abort completely. Additionally, `CreateJunction` failures when creating convenience symlinks produce cryptic error messages, though these were already non-fatal warnings. Both fail because `DeviceIoControl` returns `ERROR_INVALID_FUNCTION` when the filesystem doesn't implement `FSCTL_GET_REPARSE_POINT` or `FSCTL_SET_REPARSE_POINT` operations. **Proposed solution:** Handle `ERROR_INVALID_FUNCTION` gracefully by treating it as a "not supported" condition rather than a fatal error: 1. in `ReadSymlinkOrJunction` (`file.cc`:592): return `kNotALink` instead of `kError` when `ERROR_INVALID_FUNCTION` occurs. This allows path resolution to continue for non-symlink paths on unsupported filesystems. 2. in `CreateJunction` (`file.cc`:461): return new `kNotSupported` result code when `ERROR_INVALID_FUNCTION` occurs. This produces clear "filesystem does not support junctions" warnings instead of cryptic "Incorrect function" messages. This improves UX but doesn't change behavior (these failures were already non-fatal). This follows the try-first, fallback-on-error pattern (EAFP) used by other major projects when handling unsupported filesystem operations. **Prior art:** - Rust (rust-lang/rust#138133): checks `ERROR_INVALID_FUNCTION`, `ERROR_NOT_SUPPORTED`, and `ERROR_INVALID_PARAMETER` for filesystem operation fallbacks in `std::fs::rename`. - Microsoft STL (microsoft/STL#2077): handles junctions and reparse point errors including `ERROR_INVALID_PARAMETER` with robust fallback logic in `filesystem.cpp`. - Go (golang/go#20506): uses fallback strategies when symlink APIs are unavailable on different Windows versions. - WinFsp (winfsp/winfsp#88): documents that `ERROR_INVALID_FUNCTION` indicates `STATUS_NOT_IMPLEMENTED` for unsupported operations. - Microsoft Learn: recommends checking `FILE_SUPPORTS_REPARSE_POINTS` flag via `GetVolumeInformation`, but try-catch approach is simpler and handles edge cases where detection succeeds but operations fail. **Impact**: - enables Bazel to work on virtiofs, VirtualBox shared folders, RAM disks, and other filesystems that don't support Windows junction operations. - convenience symlinks (bazel-bin, bazel-out, etc.) still won't be created, but now with clearer error messages. **Limitations**: Full junction support would require filesystem-level changes (e.g., virtiofs driver improvements). **Testing:** Tested on Windows 11 VM with host directory mounted via virtiofs, with [rules_pkg](https://github.com/bazelbuild/rules_pkg/blob/6cdaba69ee76463b2b8e97e8d243dbb6115c3aee/toolchains/git/git_configure.bzl#L40). Before change: build analysis aborted with "Cannot read link" fatal error. After change: builds complete successfully with clearer warnings about unsupported junctions for convenience symlinks. Closes bazelbuild#27598. PiperOrigin-RevId: 833360316 Change-Id: I3751602b2bd793c1cee75b7b66fa73c955a72517
…27695) **Problem**: Bazel fails completely on Windows when using filesystems that don't support junction/reparse point operations (e.g., virtiofs, VirtualBox shared folders, network drives, RAM disks). The fatal error occurs when `ReadSymlinkOrJunction` fails during path resolution (e.g., when Starlark code calls `.realpath`): "Cannot read link: DeviceIoControl: Incorrect function". This causes build analysis to abort completely. Additionally, `CreateJunction` failures when creating convenience symlinks produce cryptic error messages, though these were already non-fatal warnings. Both fail because `DeviceIoControl` returns `ERROR_INVALID_FUNCTION` when the filesystem doesn't implement `FSCTL_GET_REPARSE_POINT` or `FSCTL_SET_REPARSE_POINT` operations. **Proposed solution:** Handle `ERROR_INVALID_FUNCTION` gracefully by treating it as a "not supported" condition rather than a fatal error: 1. in `ReadSymlinkOrJunction` (`file.cc`:592): return `kNotALink` instead of `kError` when `ERROR_INVALID_FUNCTION` occurs. This allows path resolution to continue for non-symlink paths on unsupported filesystems. 2. in `CreateJunction` (`file.cc`:461): return new `kNotSupported` result code when `ERROR_INVALID_FUNCTION` occurs. This produces clear "filesystem does not support junctions" warnings instead of cryptic "Incorrect function" messages. This improves UX but doesn't change behavior (these failures were already non-fatal). This follows the try-first, fallback-on-error pattern (EAFP) used by other major projects when handling unsupported filesystem operations. **Prior art:** - Rust (rust-lang/rust#138133): checks `ERROR_INVALID_FUNCTION`, `ERROR_NOT_SUPPORTED`, and `ERROR_INVALID_PARAMETER` for filesystem operation fallbacks in `std::fs::rename`. - Microsoft STL (microsoft/STL#2077): handles junctions and reparse point errors including `ERROR_INVALID_PARAMETER` with robust fallback logic in `filesystem.cpp`. - Go (golang/go#20506): uses fallback strategies when symlink APIs are unavailable on different Windows versions. - WinFsp (winfsp/winfsp#88): documents that `ERROR_INVALID_FUNCTION` indicates `STATUS_NOT_IMPLEMENTED` for unsupported operations. - Microsoft Learn: recommends checking `FILE_SUPPORTS_REPARSE_POINTS` flag via `GetVolumeInformation`, but try-catch approach is simpler and handles edge cases where detection succeeds but operations fail. **Impact**: - enables Bazel to work on virtiofs, VirtualBox shared folders, RAM disks, and other filesystems that don't support Windows junction operations. - convenience symlinks (bazel-bin, bazel-out, etc.) still won't be created, but now with clearer error messages. **Limitations**: Full junction support would require filesystem-level changes (e.g., virtiofs driver improvements). **Testing:** Tested on Windows 11 VM with host directory mounted via virtiofs, with [rules_pkg](https://github.com/bazelbuild/rules_pkg/blob/6cdaba69ee76463b2b8e97e8d243dbb6115c3aee/toolchains/git/git_configure.bzl#L40). Before change: build analysis aborted with "Cannot read link" fatal error. After change: builds complete successfully with clearer warnings about unsupported junctions for convenience symlinks. Closes #27598. PiperOrigin-RevId: 833360316 Change-Id: I3751602b2bd793c1cee75b7b66fa73c955a72517 Commit dab96fc Co-authored-by: Rgis Desgroppes <rdesgroppes@gmail.com>
|
The changes in this PR have been included in Bazel 9.0.0 RC3. Please test out the release candidate and report any issues as soon as possible. |
Problem:
Bazel fails completely on Windows when using filesystems that don't support junction/reparse point operations (e.g., virtiofs, VirtualBox shared folders, network drives, RAM disks).
The fatal error occurs when
ReadSymlinkOrJunctionfails during path resolution (e.g., when Starlark code calls.realpath): "Cannot read link: DeviceIoControl: Incorrect function".This causes build analysis to abort completely.
Additionally,
CreateJunctionfailures when creating convenience symlinks produce cryptic error messages, though these were already non-fatal warnings.Both fail because
DeviceIoControlreturnsERROR_INVALID_FUNCTIONwhen the filesystem doesn't implementFSCTL_GET_REPARSE_POINTorFSCTL_SET_REPARSE_POINToperations.Proposed solution:
Handle
ERROR_INVALID_FUNCTIONgracefully by treating it as a "not supported" condition rather than a fatal error:ReadSymlinkOrJunction(file.cc:592): returnkNotALinkinstead ofkErrorwhenERROR_INVALID_FUNCTIONoccurs. This allows path resolution to continue for non-symlink paths on unsupported filesystems.CreateJunction(file.cc:461): return newkNotSupportedresult code whenERROR_INVALID_FUNCTIONoccurs. This produces clear "filesystem does not support junctions" warnings instead of cryptic "Incorrect function" messages. This improves UX but doesn't change behavior (these failures were already non-fatal).This follows the try-first, fallback-on-error pattern (EAFP) used by other major projects when handling unsupported filesystem operations.
Prior art:
ERROR_INVALID_FUNCTION,ERROR_NOT_SUPPORTED, andERROR_INVALID_PARAMETERfor filesystem operation fallbacks instd::fs::rename.<filesystem>:read_symlinkdoes not work with junctions microsoft/STL#2077): handles junctions and reparse point errors includingERROR_INVALID_PARAMETERwith robust fallback logic infilesystem.cpp.ERROR_INVALID_FUNCTIONindicatesSTATUS_NOT_IMPLEMENTEDfor unsupported operations.FILE_SUPPORTS_REPARSE_POINTSflag viaGetVolumeInformation, but try-catch approach is simpler and handles edge cases where detection succeeds but operations fail.Impact:
Limitations:
Full junction support would require filesystem-level changes (e.g., virtiofs driver improvements).
Testing:
Tested on Windows 11 VM with host directory mounted via virtiofs, with rules_pkg. Before change: build analysis aborted with "Cannot read link" fatal error. After change: builds complete successfully with clearer warnings about unsupported junctions for convenience symlinks.