The use of .data is deprecated in PyTorch. It's recommended to use .detach().clone() instead to avoid potential issues with autograd and ensure proper tensor handling.

The try...except block is too broad. It's generally better to catch specific exceptions like OSError or IOError for file operations, rather than a generic Exception. This helps in debugging and understanding the root cause of failures.

Executing install_to_cache(__file__, "moe_utils.py") directly at the module level means this code runs every time moe_utils.py is imported. While this might be intended for setup, it can lead to unexpected side effects or performance overhead in certain scenarios (e.g., repeated imports, testing environments). Consider wrapping this call in a function or a conditional block (e.g., if __name__ == "__main__":) if it's meant for a specific initialization step, or add a comment explaining why it's necessary to run at import time.

The try...except Exception block is too broad. For a runtime check like this, it would be more precise to catch specific exceptions that torch._grouped_mm might raise, such as RuntimeError or AttributeError, to avoid masking other potential issues.

The try...except Exception block is too broad. When dealing with external libraries like Triton, it's usually more appropriate to catch ImportError or ModuleNotFoundError if the issue is related to the library's availability, or specific runtime errors if the issue is with its usage.

The try...except Exception block is too broad. When extracting LoRA weights, it's better to catch specific exceptions that might occur during attribute access or tensor manipulation, such as AttributeError, KeyError, or IndexError, to provide more targeted error handling and debugging.

Importing forward_native_grouped_mm, forward_triton_grouped_gemm, and forward_native_moe_loop locally within the old_forward function is unusual. While it works, it can make the code harder to read and potentially lead to subtle scoping issues or unexpected behavior if the module structure changes. It's generally better practice to place imports at the top of the module or function scope where they are first needed, rather than inside a dynamically patched function. Consider moving these imports to the module level if they are consistently used across different patched functions.

Similar to the old_forward function, importing forward_native_grouped_mm, forward_triton_grouped_gemm, and forward_native_moe_loop locally within this forward function is not ideal. It's generally better practice to place imports at the top of the module or function scope where they are first needed. For dynamically patched functions, this pattern might be a workaround, but it could impact readability and maintainability. If these functions are meant to be globally available to the patched methods, module-level imports would be clearer.

Fix MoE 4-bit dim inference for Qwen3-VL layout

The quantized module infers intermediate_dim and hidden_dim from gate_up_proj.shape[1]/[2], which assumes the standard (E, 2I, H) layout. This same commit patches Qwen3‑VL MoE to store weights in grouped_mm format (E, H, 2I) in qwen3_vl_moe.py (lines 189–231), so for Qwen3‑VL this calculation swaps dimensions (intermediate_dim becomes H/2 and hidden_dim becomes 2*I). The result is a quantized module with wrong shapes, which will fail to load weights or produce invalid outputs. Consider detecting the transposed layout or using config.hidden_size/moe_intermediate_size instead of raw tensor dims.

Useful? React with 👍 / 👎.

Make 4-bit MoE save/load key scheme consistent

The custom _save_to_state_dict writes weights under gate_up_projs.*/down_projs.*, but _load_from_state_dict only recognizes the stacked gate_up_proj/down_proj keys and otherwise defers to the default loader (which expects _bnb_gate_up_weights.*). A checkpoint saved by this module will therefore reload with missing quantized weights (keys become unexpected and the ParameterLists stay empty), breaking inference on reload. The save/load key names need to align so round‑trip works.

Useful? React with 👍 / 👎.