Skip to content

Improve progress bar new#440

Merged
joein merged 8 commits into
mainfrom
improve-progress-bar-new
Jan 27, 2025
Merged

Improve progress bar new#440
joein merged 8 commits into
mainfrom
improve-progress-bar-new

Conversation

@hh-space-invader

Copy link
Copy Markdown
Contributor

All Submissions:

  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same update/change?

New Feature Submissions:

  • Does your submission pass the existing tests?
  • Have you added tests for your feature?
  • Have you installed pre-commit with pip3 install pre-commit and set up hooks with pre-commit install?

New models submission:

  • Have you added an explanation of why it's important to include this model?
  • Have you added tests for the new model? Were canonical values for tests computed via the original model?
  • Have you added the code snippet for how canonical values were computed?
  • Have you successfully ran tests with your changes locally?

Comment thread fastembed/common/model_management.py Outdated
Comment on lines +166 to +167
if not metadata: # happens when internet connection is down while downloading
raise ValueError("No valid files found to save metadata")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this exception is not required, since it is an internal function and exception occurs due to another important problem which will be seen anyway

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one is actually required. If the network connection is down while downloading, then there will be no raise at all, and then we will get failed to load model rather couldn't download model from any sources.
The reason is, snapshot_download returns string either succeeded or failed. Hence, I had to raise while saving to make sure everything downloaded successfully.

Comment thread fastembed/common/model_management.py Outdated
metadata_file = snapshot_dir / cls.METADATA_FILE

try:
# at least there's network first time (we should make sure to not fail if no network)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure if I understand this comment, it looks like "at least there's network first time" is from the previous iteration

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

model_info and list_repo_tree are network calls to HF. So at least we would have network while downloading first time.

@joein joein force-pushed the improve-progress-bar-new branch from 2e07ced to 3928f3f Compare January 27, 2025 10:31
@joein joein merged commit 54f6cd9 into main Jan 27, 2025
@joein joein deleted the improve-progress-bar-new branch January 27, 2025 21:34
@joein joein mentioned this pull request Jan 27, 2025
9 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants