Handle symlinks in LocalFileSystem (#2206) #2269
Conversation
| pub(crate) fn filesystem_path_to_url( | ||
| path: impl AsRef<std::path::Path>, | ||
| ) -> Result<Url, Error> { | ||
| let path = path.as_ref().canonicalize().context(CanonicalizeSnafu { |
There was a problem hiding this comment.
This is the change that makes this possible, we no longer canonicalize the paths
| let walkdir = WalkDir::new(&resolved_prefix) | ||
| .min_depth(1) | ||
| .max_depth(1) | ||
| .follow_links(true); |
There was a problem hiding this comment.
Amusingly the lack of follow_links previously meant that symlinks would be reported as objects even if they were actually directories 😅
| /// If not called from a tokio context, this will perform IO on the current thread with | ||
| /// no additional complexity or overheads | ||
| /// | ||
| /// # Symlinks |
| .await | ||
| .unwrap(); | ||
|
|
||
| // Follow out of tree symlink |
There was a problem hiding this comment.
love the comments in this test
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
|
Benchmark runs are scheduled for baseline = 4222f5a and contender = ad65e88. ad65e88 is a master commit associated with this PR. Results will be available as each benchmark for each run completes. |
* Handle symlinks in LocalFileSystem (#2206) * Update object_store/src/local.rs Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org> Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Which issue does this PR close?
Closes apache/arrow-rs-object-store#245
Rationale for this change
What changes are included in this PR?
This attempts to handle symlinks consistently
Are there any user-facing changes?
Yes, symlinks will behave as expected