Skip to content

llmcompressor.transformers

Tools for integrating LLM Compressor with transformers training flows.

Modules:

Functions:

is_model_ct_quantized_from_path 

is_model_ct_quantized_from_path(path: str) -> bool

Determine if model from path is quantized based on the config

Parameters:

  • path

    (str) –

    path to the model or HF stub

Returns:

  • bool –

    True if config contains quantization_config from the given path

Source code in llmcompressor/transformers/utils/helpers.py 
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
def is_model_ct_quantized_from_path(path: str) -> bool:
    """
    Determine if model from path is quantized based
    on the config

    :param path: path to the model or HF stub
    :return: True if config contains quantization_config from the given path

    """
    config = AutoConfig.from_pretrained(path)
    if config is not None:
        if (
            hasattr(config, "quantization_config")
            and config.quantization_config["quant_method"] == "compressed-tensors"
        ):
            return True
    return False