[Model] Add HyperCLOVAX-SEED-Think-14B language model support#37107
Conversation
|
Documentation preview: https://vllm--37107.org.readthedocs.build/en/37107/ |
There was a problem hiding this comment.
Code Review
This pull request adds support for the HyperCLOVAX-SEED-Think-14B language model. The changes include a new model implementation, a corresponding configuration class, and updates to the model registries, documentation, and tests. The new implementation handles the model's specific architectural features, such as muP scaling and optional Peri-Layer Normalization. The code is well-structured and follows existing patterns in the vLLM codebase. One issue was found in the test registry update, where a redundant and incorrect entry was added.
Signed-off-by: bigshanedogg <bigshane319@gmail.com>
Signed-off-by: bigshanedogg <bigshane319@gmail.com>
Signed-off-by: bigshanedogg <bigshane319@gmail.com>
fe13838 to
71bc1b9
Compare
…roject#37107) Signed-off-by: bigshanedogg <bigshane319@gmail.com>
…roject#37107) Signed-off-by: bigshanedogg <bigshane319@gmail.com>
…roject#37107) Signed-off-by: bigshanedogg <bigshane319@gmail.com>
…roject#37107) Signed-off-by: bigshanedogg <bigshane319@gmail.com>
…roject#37107) Signed-off-by: bigshanedogg <bigshane319@gmail.com>
…roject#37107) Signed-off-by: bigshanedogg <bigshane319@gmail.com>
…roject#37107) Signed-off-by: bigshanedogg <bigshane319@gmail.com>
…roject#37107) Signed-off-by: bigshanedogg <bigshane319@gmail.com>
…roject#37107) Signed-off-by: bigshanedogg <bigshane319@gmail.com>
|
@bigshanedogg |
…roject#37107) Signed-off-by: bigshanedogg <bigshane319@gmail.com> Signed-off-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Purpose
Add inference support for HyperCLOVA X (
HyperCLOVAXForCausalLM), a large language model family developed by NAVER Cloud.Changes
vllm/model_executor/models/hyperclovax.py(new) —HyperCLOVAXForCausalLMmodel implementationvllm/transformers_utils/configs/hyperclovax.py(new) —HyperCLOVAXConfigconfiguration classvllm/model_executor/models/registry.py— RegisterHyperCLOVAXForCausalLMvllm/transformers_utils/configs/__init__.py— RegisterHyperCLOVAXConfigdocs/models/supported_models.md— AddHyperCLOVAXForCausalLMentrytests/models/registry.py— Add test registry entry (naver-hyperclovax/HyperCLOVAX-SEED-Think-14B)tests/models/language/generation/test_common.py— AddHyperCLOVAXForCausalLMto common generation testsTest Plan
Launch server
vllm serve naver-hyperclovax/HyperCLOVAX-SEED-Think-14B \ --max-model-len 32768 \ --max-num-batched-tokens 16384 \ --tensor-parallel-size 1 \ --trust-remote-code \ --enable-prefix-cachingTest Result
Benchmark validation
Request
client
output