#model-optimization

Articles tagged with #model-optimization

Cache Eviction
The key-value cache is an inference optimization technique that eliminates redundant recomputation of past token representations during autoregressive generation. The KV cache memory footprint scales
Aug 6, 20269 min read6
Model Quantization
Quantization is a model compression technique that reduces numerical precision of weights and activations from floating-point to lower-bit representations, decreasing model size and computational cost
Jul 31, 20265 min read1
Neural Architecture Search
Automates the process of determining optimal model configurations by systematically exploring large spaces of possible architecture to identify those that best balance accuracy, computational cost, me
Jul 30, 20263 min read
Knowledge distillation
Knowledge distillation involves using a large teacher model to train a smaller student model. The student model not only learns from the correct labels but also from the teacher's output distribution.
Jul 27, 20263 min read1
Model Pruning
Structured model optimization Structured model optimization works in two key ways: Eliminating parameter redundancy. Structuring computations for efficient hardware execution through techniques like
Jul 25, 20263 min read5