Clarify Kubernetes version requirement and fallback plan in Key Features #380
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kerthcet
reviewed
Apr 25, 2025
| - **Accelerator Fungibility**: llmaz supports serving the same LLM with various accelerators to optimize cost and performance. | ||
| - **Various Model Providers**: llmaz supports a wide range of model providers, such as [HuggingFace](https://huggingface.co/), [ModelScope](https://www.modelscope.cn), ObjectStores. llmaz will automatically handle the model loading, requiring no effort from users. | ||
| - **Multi-Host Support**: llmaz supports both single-host and multi-host scenarios with [LWS](https://github.com/kubernetes-sigs/lws) from day 0. | ||
| - **Multi-Host Support**: llmaz supports both single-host and multi-host scenarios with [LWS](https://github.com/kubernetes-sigs/lws) from day 0. **Important**: LWS requires Kubernetes version **v1.26 or higher**. If you are using a lower Kubernetes version and most of your workloads rely on single-node inference, we may consider replacing LWS with a deployment-based approach. This fallback plan would involve using Kubernetes Deployments to manage single-node inference workloads efficiently. See [#32](https://github.com/InftyAI/llmaz/issues/32) for more details and updates. |
Member
There was a problem hiding this comment.
Could you update this in https://github.com/InftyAI/llmaz/blob/main/docs/installation.md#prerequisites? Thanks
Contributor
Author
There was a problem hiding this comment.
Sure. I will complete it and make a commit.
Member
There was a problem hiding this comment.
Please revert this change, the note within installation.md is enough. Thanks
Member
|
/kind documentation |
Contributor
Author
|
I've updated the Installation file. Can you please review it? |
kerthcet
reviewed
Apr 26, 2025
| - **Accelerator Fungibility**: llmaz supports serving the same LLM with various accelerators to optimize cost and performance. | ||
| - **Various Model Providers**: llmaz supports a wide range of model providers, such as [HuggingFace](https://huggingface.co/), [ModelScope](https://www.modelscope.cn), ObjectStores. llmaz will automatically handle the model loading, requiring no effort from users. | ||
| - **Multi-Host Support**: llmaz supports both single-host and multi-host scenarios with [LWS](https://github.com/kubernetes-sigs/lws) from day 0. | ||
| - **Multi-Host Support**: llmaz supports both single-host and multi-host scenarios with [LWS](https://github.com/kubernetes-sigs/lws) from day 0. **Important**: LWS requires Kubernetes version **v1.26 or higher**. If you are using a lower Kubernetes version and most of your workloads rely on single-node inference, we may consider replacing LWS with a deployment-based approach. This fallback plan would involve using Kubernetes Deployments to manage single-node inference workloads efficiently. See [#32](https://github.com/InftyAI/llmaz/issues/32) for more details and updates. |
Member
There was a problem hiding this comment.
Please revert this change, the note within installation.md is enough. Thanks
Member
|
/lgtm Thanks for your patience. Welcome onboard! |
Contributor
Author
|
Thank you! I appreciate the review and approval. Excited to contribute further! |
Member
|
/lgtm |
Member
|
seems we need to rebase. |
Co-authored-by: Kante Yin <kerthcet@gmail.com>
971cf5e to
15fee24
Compare
Contributor
Author
|
Are there any final changes required? Please clarify, and I will make the necessary updates. |
Member
|
/lgtm |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
…res section
What this PR does / why we need it
This PR updates the README file to improve clarity on multiple host support and installation guidelines by specifying Kubernetes version requirements, helping to prevent setup issues.
Which issue(s) this PR fixes
Fixes #
#379
Special notes for your reviewer
Does this PR introduce a user-facing change?