Skip to main content

Ctrl+K

LLMC

Quick Start

Installing LLMC

Configs

Configs’ brief description

Advanced

Model accuracy test V1
Model accuracy test V2
Custom calibration datasets
Vit quant and img datatsets
VLM Quantization and custom_mm Datasets
Layerwise mixed bits quantization
Model Sparsification
Token Reduction

Best Practice

AWQ
AWQ + OmniQuant
QuaRot + GPTQ

Backbend

VLLM Quantized Inference
SGLang Quantized Inference
AutoAWQ Quantized Inference
MLC LLM Quantized Inference

Search

Ctrl+K

By ModelTC

© Copyright 2024, llmc contributors.