Slide
Quantize What Counts: More for Keys, Less for Values
ACL 2026 presentation on Quantize What Counts: More for Keys, Less for Values, explaining key-value norm disparity, key-prioritized quantization, and practical KV-cache compression guidance.
Quantization
Tag: Quantization
6 items tagged with "Quantization"
Slide
Serving Reasoning LLMs Efficiently and Reliably [No Anime]
Serving reasoning LLMs efficiently and reliably: lossless DFloat11 compression, KV-cache quantization, and Bayes@N evaluation and ranking under test-time scaling.
Slide
Serving Reasoning LLMs Efficiently and Reliably
Serving reasoning LLMs efficiently and reliably: lossless DFloat11 compression, KV-cache quantization, and Bayes@N evaluation and ranking under test-time scaling.
Poster
Quantize What Counts: More for Keys, Less for Values
Mohsen Hariri, Alan Luo, Weicong Chen, Tianyi Zhang, Qifan Wang, Xiaotian Han, Vipin Chaudhary
acl-2026
A geometry-driven mixed-precision KV-cache quantization poster showing that keys carry more information than values, so key-favored bit allocation preserves accuracy while reducing memory.
Slide
Test-Time Scaling Under Budget
M.Sc. thesis defense on evaluating and ranking reasoning LLMs under a fixed test-time sampling budget, covering Pass@k's instability, Bayesian alternatives, and how many samples a reliable ranking actually needs.
Paper
Quantize What Counts: More for Keys, Less for Values
Mohsen Hariri, Alan Luo, Weicong Chen, Tianyi Zhang, Qifan Wang, Xiaotian Han, Vipin Chaudhary
Key-favored KV-cache quantization for LLMs: theory shows keys have larger norms and should get more bits; empirics show 4b-K/2b-V preserves up to 98.3% accuracy while cutting memory.