dev(narugo): add write_bytes functions to hfutils.index system#66
dev(narugo): add write_bytes functions to hfutils.index system#66narugo1992 merged 16 commits intomainfrom
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #66 +/- ##
==========================================
+ Coverage 87.20% 87.84% +0.63%
==========================================
Files 63 64 +1
Lines 2150 2304 +154
==========================================
+ Hits 1875 2024 +149
- Misses 275 280 +5
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Copilot reviewed 3 out of 4 changed files in this pull request and generated 1 comment.
Files not reviewed (1)
- docs/source/api_doc/index/local_fetch.rst: Language not supported
Comments suppressed due to low confidence (1)
hfutils/index/local_fetch.py:319
- [nitpick] The parameter name 'file' shadows a Python built-in. Consider renaming it to 'file_obj' to avoid potential confusion.
def tar_file_write_bytes(archive_file: str, file_in_archive: str, file: BinaryIO,
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Copilot reviewed 3 out of 4 changed files in this pull request and generated no comments.
Files not reviewed (1)
- docs/source/api_doc/index/local_fetch.rst: Language not supported
Comments suppressed due to low confidence (1)
hfutils/index/local_fetch.py:424
- Setting no_validate to True in the tar_file_download function bypasses size/hash verification, which is inconsistent with the behavior in tar_file_write_bytes. Please confirm that this design choice is intentional to avoid potential integrity issues.
no_validate=True,
|
now waiting for huggingface/huggingface_hub#2978 and huggingface/huggingface_hub#2979 after hf_transfer explicit option is on, we can go into next step create a binaryio class which only implement then write and tell function, so that we can proxy the binary writing and validate its binary sha256 |
There was a problem hiding this comment.
Copilot reviewed 8 out of 12 changed files in this pull request and generated no comments.
Files not reviewed (4)
- docs/source/api_doc/index/fetch.rst: Language not supported
- docs/source/api_doc/index/local_fetch.rst: Language not supported
- docs/source/api_doc/utils/binary_proxy.rst: Language not supported
- docs/source/api_doc/utils/index.rst: Language not supported
No description provided.