The current implementation of load_weights reuses Qwen3ForCausalLM.load_weights. However, the weight loading logic in Qwen3ForCausalLM has a special case for "Embedding" models that prefixes all weights with model.. This is incorrect for the score layer in Qwen3ForSequenceClassification, which is a top-level attribute, not part of the model submodule. As a result, the weights for the classification head will fail to load, leading to a model that produces incorrect outputs.

To fix this, implement a custom load_weights implementation that correctly handles the weight paths for the score layer.

    def load_weights(self, weights: Iterable[Tuple[str, torch.Tensor]]):
        # HACK: Temporarily modify config to avoid incorrect prefixing in Qwen3ForCausalLM.load_weights
        # for the 'score' layer, which is not under the 'model' submodule.
        original_name_or_path = self.config.name_or_path
        try:
            self.config.name_or_path = ""  # Disable the "Embedding" check in the reused loader

            prefixed_weights = []
            for name, w in weights:
                if name.startswith("lm_head"):
                    continue

                if not name.startswith("score."):
                    prefixed_weights.append((f"model.{name}", w))
                else:
                    prefixed_weights.append((name, w))

            return Qwen3ForCausalLM.load_weights(self, prefixed_weights)
        finally:
            # Restore config
            self.config.name_or_path = original_name_or_path

The model class in parenthesis (QwenEmbeddingModel) and the description "compatible with QwenEmbeddingModel interface" may be misleading for the new Qwen3 model. The actual class implemented is Qwen3ForSequenceClassification. To improve clarity for users, update the model class and description to accurately reflect the new implementation.

could you give a reference here?

For the reference request on lines 38-44:

Thank you for the feedback. I've added a specific reference to the official Qwen3-Embedding implementation:

# Use normalize=True for qwen3 embedding based on official implementation
# Reference: https://github.com/QwenLM/Qwen3-Embedding/blob/main/examples/qwen3_embedding_transformers.py#L55
# Official code: output = F.normalize(output, p=2, dim=1)

This references the official transformers implementation where they use F.normalize(output, p=2, dim=1) for L2 normalization, which corresponds to our normalize=True setting in the pooler.

It seems this logic depends on whether we use embedding model or rerank model. This logic is right for embedding model, it looks like in rerank model it uses softmax in pooler, could we change pooler normalize or softmax according to model architectures?

could we add ut in test_embedding_models.py as Qwen2ForSequenceClassification does?

For adding Qwen3ForSequenceClassification to is_generation_model unit tests:

I've made the following changes:

1. Added unit tests: Added ("Qwen/Qwen3-Embedding-8B", 1, 1e-5) to the MODELS list in test/srt/models/test_embedding_models.py, following the same pattern as Qwen2ForSequenceClassification models.

could you provide more details about how to use Qwen3ForSequenceClassification as vllm-project/vllm#19260 does?