Skip to content

llmcompressor.modeling.qwen3_next_moe

Classes:

CalibrationQwen3NextSparseMoeBlock 

CalibrationQwen3NextSparseMoeBlock(
    original: Qwen3NextSparseMoeBlock,
    config: Qwen3NextConfig,
    calibrate_all_experts: bool = True,
)

Bases: MoECalibrationModule

Calibration version of Qwen3NextSparseMoeBlock that sends all tokens to all experts.

Source code in llmcompressor/modeling/qwen3_next_moe.py 
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
def __init__(
    self,
    original: Qwen3NextSparseMoeBlock,
    config: Qwen3NextConfig,
    calibrate_all_experts: bool = True,
):
    super().__init__()
    self.num_experts = config.num_experts
    self.top_k = config.num_experts_per_tok
    self.norm_topk_prob = config.norm_topk_prob

    # gating
    self.calibrate_all_experts = calibrate_all_experts
    self.gate = original.gate
    self.experts = original.experts

    self.shared_expert = original.shared_expert
    self.shared_expert_gate = original.shared_expert_gate