Checklist

• 1. If the issue you raised is not a feature but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
• 2. Please use English, otherwise it will be closed.

Motivation

Description

It would be beneficial to introduce model hooks that allow users to access and modify model activations. This feature would enable greater flexibility for tasks such as visualization, debugging, and custom processing of intermediate representations.

Use case

• Extract intermediate outputs for interpretability analysis, such as LogitLens-style investigations.
• Expose internal activations, enabling users to cache activations and implement functions to edit, remove, or replace them dynamically during inference, for example representation engineering.

While this may introduce some performance overhead, it would enhance interpretability research and enable efficient model editing.

Related resources

model hook resources

• Pytorch hook
• Transformerlens

related issues and use case

• [Usage]: How do we add model hooks?  vllm-project/vllm#4084
• [Feature]: obtain logits vllm-project/vllm#11397
• [Usage]: How to access mlp layer using the current version vllm(0.4.0) vllm-project/vllm#8278