我按照https://github.com/kvcache-ai/Mooncake/blob/main/doc/zh/vllm-integration-v1.md 这个教程能把服务启动了但是在测试的时候 decode instance 报错
INFO 06-16 09:49:16 [launcher.py:36] Route: /metrics, Methods: GET INFO: Started server process [11506] INFO: Waiting for application startup. INFO: Application startup complete. INFO 06-16 09:49:30 [logger.py:39] Received request cmpl-807b5ad07b884846a8ad27608a20a3f2-0: prompt: 'San Francisco is a', params: SamplingParams(n=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.6, top_p=0.95, top_k=20, min_p=0.0, seed=None, stop=[], stop_token_ids=[], bad_words=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=100, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None, guided_decoding=None, extra_args=None), prompt_token_ids: [23729, 12879, 374, 264], lora_request: None, prompt_adapter_request: None. INFO 06-16 09:49:30 [engine.py:313] Added request cmpl-807b5ad07b884846a8ad27608a20a3f2-0. python3: /home/runner/work/Mooncake/Mooncake/mooncake-store/src/transfer_task.cpp:164: void mooncake::TransferEngineOperationState::set_result_internal(mooncake::ErrorCode): Assertion!result_.has_value() && "Result should only be set once."' failed.
ERROR 06-16 09:49:36 [client.py:306] RuntimeError('Engine process (pid 11809) died.')
ERROR 06-16 09:49:36 [client.py:306] NoneType: None
INFO: Shutting down
INFO: Waiting for connections to close. (CTRL+C to force quit)
INFO: 127.0.0.1:51976 - "POST /v1/completions HTTP/1.1" 500 Internal Server Error
INFO: Waiting for application shutdown.
INFO: Application shutdown complete.
INFO: Finished server process [11506]
/usr/lib/python3.12/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
`
单独对decode/prefill instance 测试都是没问题的

config.json配置
{ "local_hostname": "33.254.xxx.xxx", "metadata_server": "http://33.254.xxx.xxx:8080/metadata", "protocol": "tcp", "device_name": "", "master_server_address": "33.254.xxx.xxx:50001" }

启动服务命令:
MOONCAKE_CONFIG_PATH=./config.json VLLM_USE_V1=0 python3 -m vllm.entrypoints.openai.api_server --model /model/qwen3-8b/ --port 8100 --max-model-len 10000 --gpu-memory-utilization 0.8 --kv-transfer-config '{"kv_connector":"MooncakeStoreConnector","kv_role":"kv_producer"}'

MOONCAKE_CONFIG_PATH=./config.json CUDA_VISIBLE_DEVICES=1 MOONCAKE_CONFIG_PATH=./config.json VLLM_USE_V1=0 python3 -m vllm.entrypoints.openai.api_server --model /model/qwen3-8b/ --port 8200 --max-model-len 10000 --gpu-memory-utilization 0.8 --kv-transfer-config '{"kv_connector":"MooncakeStoreConnector","kv_role":"kv_consumer"}'

core dumped #495

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions